Commit Graph

3348 Commits

Author SHA1 Message Date
zah 55ae7d3b00
Version 23.4.0 (#4857)
* Draft release notes for version 23.4.0

* Drop the reference to the Builder API failsafe spec PR
2023-04-25 22:46:38 +03:00
henridf da6169bfe9
BlockProcessor.storeBlock: write blobs to DB (#4854) 2023-04-25 13:55:35 +03:00
zah 9b2c07c118
Allow exiting multiple validators at once (#4855) 2023-04-25 09:44:01 +03:00
Jacek Sieka 58b93ccbe0
era: Capella+ support (fixes #4752) (#4853)
Post-Capella, historical roots are computed from historical summaries
instead of being directly stored in the beacon state.

Slightly messy to pass both lists around - this is done to avoid
computing the historical root unnecessarily.
2023-04-24 15:26:28 +02:00
tersec fbe90dcbea
update statediffs to support capella (#4852) 2023-04-23 22:15:14 +03:00
tersec ed7ad56d16
allow for biasing comparisons between engine/builder api bids (#4847) 2023-04-23 22:10:34 +03:00
tersec b390911e93
automatic update of v1.3.0-rc.5 consensus spec URLs to v1.3.0 (#4848) 2023-04-21 18:52:43 +00:00
tersec c0b90833db
don't crash on block production error; build lookahead proposal from current head (#4842) 2023-04-20 15:07:27 +00:00
tersec d3400ca11b
low attestations during epoch should instafail in CI; dbg -> warn level in `newPayload` log (#4830)
* low attestations during epoch should instafail in CI; dbg -> warn level on newPayload log

* improve newPayload warning message when no valid EL connected

* reduce potential spam; make log spelling more consistent; use fatal/quit
2023-04-19 19:42:30 +00:00
tersec 2246a6ec95
Revert "include small dedup in block processor to handle blockByRoot blocks (#4814)" (#4840)
This reverts commit 8b3ffec0d5.

Syncing was broken with this: https://github.com/status-im/infra-nimbus/issues/132#issuecomment-1514465481
2023-04-19 19:16:27 +00:00
henridf 12d640b691
Request manager: Handle blobless blocks (#4833)
Post-Deneb, when the request manager receives a missing block from a
peer, it needs to check if the corresponding blobs are available, and
if so pass them along. If they aren't available, the newly-fetched
block must be put in blobless quarantine (while the blobs are
retrieved, coming in next commit).
2023-04-19 19:37:38 +03:00
tersec 0fc89484de
ignore EL exchangeTransitionConfiguration results (#4831) 2023-04-19 12:16:50 +00:00
tersec 4643a357ef
consensus spec ref URL update to v1.3.0-rc.5 (#4837) 2023-04-18 22:00:06 +00:00
tersec 45f89f1205
use consensus specs v1.3.0 test vectors (#4838) 2023-04-18 22:01:49 +02:00
Etan Kissling e6e4ba9de6
clean up redundant tests and config (#4836)
The consensus-spec-tests already cover the scenarios of our custom test
runner, so the custom tests can be removed. Also cleans up unused config
flags and related unreachable logic.
2023-04-18 21:26:36 +02:00
tersec 77c0aa713e
use non-nil head corresponding to execution head (#4835) 2023-04-18 14:17:56 +00:00
Etan Kissling 1d3e838236
lower log level of LC optimistic sync (#4834)
When using light client sync while main forward sync is behind, use
DEBUG log level instead of INFO log level to reduce log verbosity.
2023-04-18 13:37:29 +00:00
tersec cc24429828
remove empty block fallback now that capella's on mainnnet (#4821)
* remove empty block fallback now that capella's on mainnnet

* build_empty_execution_payload is only testing infrastructure now

* update error message
2023-04-18 09:21:15 +00:00
henridf 8f2f40f0af
Remove blockBlobsVerifier (#4829)
Having a separate 'blockBlobsVerifier' function for post-Deneb blocks
is no longer of any benefit. This commit removes it.
2023-04-18 02:12:57 +02:00
Eugene Kabanov b51152153a
VC: Hardening and optimizing time handling. (#4743)
* Fix durationToNextSlot() and durationToNextEpoch() to work not only after Genesis, but also before Genesis.
Change VC pre-genesis behavior, add runPreGenesisWaitingLoop() and runGenesisWaitingLoop().
Add checkedWaitForSlot() and checkedWaitForNextSlot() to strictly check current time and print warnings.
Fix VC main loop to use checkedWaitForNextSlot().
Fix attestation_service to run attestations processing only until the end of the duty slot.
Change attestation_service main loop to use checkedWaitForNextSlot().
Change block_service to properly cancel all the pending proposer tasks.
Use checkedWaitForSlot to wait for block proposal.
Fix block_service waitForBlockPublished() to be compatible with BN.
Fix sync_committee_service to avoid asyncSpawn.
Fix sync_committee_service to run only until the end of the duty slot.
Fix sync_committee_service to use checkedWaitForNextSlot().

* Refactor validator logging.
Fix aggregated attestation publishing missing delay.

* Fix doppelganger detection should not start at pre-genesis time.
Fix fallback service sync status spam.
Fix false `sync committee subnets subscription error`.

* Address review comments part 1.

* Address review comments.

* Fix condition issue for near genesis waiting loop.

* Address review comments.

* Address review comments 2.
2023-04-17 21:31:54 +00:00
tersec 228e10f1d9
update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
tersec 8b3ffec0d5
include small dedup in block processor to handle blockByRoot blocks (#4814) 2023-04-17 19:36:15 +00:00
Jacek Sieka 7df75d77fa
move spec constants to spec (#4825)
* move spec constants to spec

quite funny, this removes >_1000_ transitive imports when building
`nimbus_validator_client`.

`genDepend`:

```sh
[arnetheduck@praeceps beacon_chain]$ diff -u
nimbus_validator_client.dot.pre nimbus_validator_client.dot | wc
   1167    3507  103822
```

* imports

* better constants

* oops

* too many blobs

* reorder spec constants in fork order

* cleanup
2023-04-17 17:18:54 +00:00
henridf 176c80a3d5
Fix incorrect naming and use of Quarantine.peekBlobless (#4826)
The 'peek' name was incorrect as it was actually removing from the
table. It was consequently used incorrectly in block processing: the
blobless block wasn't returned to the table when it should be.
2023-04-17 18:10:13 +02:00
tersec 75be7d267d
always use fcUV2 in shapella even for non-proposer fcUs (#4817)
* always use fcUV2 in shapella even for non-proposer fcUs

* avoid template/proc naming conflict with libp2p/signed_envelope.nim having a payload proc
2023-04-17 16:17:52 +02:00
henridf 4df851f4f5
Fix incorrect handling of blobless blocks (#4822)
We were passing along all blocks to local processing, including those
for which we were missing blobs. This commit fixes that.
2023-04-17 10:54:04 +00:00
henridf 29b431e312
Simplify block quarantine blobless (#4824)
* Simplify block quarantine blobless

The quarantine blobless table was initially keyed off of (Eth2Digest,
ValidatorSig). This was modelled off the orphan table. The presence of
the signature in the key is necessary for orphans, because we can't
verify the signature for an orphan. That is not the case for a
blobless block, where the signature can be verified.

So this PR changes the blobless block table to be keyed off a
Eth2Digest only. This simplifies the retrieval and handling of
blobless blocks.

* review feedback
2023-04-16 08:37:56 +00:00
Etan Kissling cb9e0eed49
allow trusted node sync based on LC trusted block root (#4736)
* allow trusted node sync based on LC trusted block root

Extends `trustedNodeSync` with a new `--trusted-block-root` option that
allows initializing a light client. No `--state-id` must be provided.
The beacon node will then use this light client to obtain the latest
finalized state from the remote server in a trust-minimized fashion.
Note that the provided `--trusted-block-root` should be somewhat recent,
and that security precautions such as comparing the state root against
block explorers is still recommended.

* fix

* workaround for `valueOr` limitations

* reduce magic numbers

* digest len > context len for readability

* move `cstring` conversion to caller

* avoid abbreviations

* `return` codestyle
2023-04-16 06:07:07 +00:00
henridf 57623af36a
Remove unnecessary field derefs in BlockProcessor.storeBlock (#4823) 2023-04-16 01:25:17 +00:00
tersec f6062a2be1
compute correct withdrawals for lookahead proposal fcUs at epoch boundaries (#4820) 2023-04-15 21:48:24 +00:00
tersec ae45e11f8e
log when attempting to validate with exited validators (#4812) 2023-04-15 21:31:24 +00:00
henridf 021de18e06
Quarantine and reassembly of gossiped blobs and blocks (#4808) 2023-04-13 19:11:40 +00:00
Etan Kissling fa3655527c
add `formatIt` for `ForkedLightClientXyz` (#4816)
* add `formatIt` for `ForkedLightClientXyz`

Even though the `ForkyLightClientXyz` have `formatIt`, they do not apply
when logging `ForkedLightClientXyz`, leading to large logs at times.
Defining `formatIt` for `ForkedLightClientXyz` fixes this.

* exports not needed
2023-04-13 15:26:17 +00:00
Miran 0ec7df1042
fix spelling in conf.nim (#4813) 2023-04-12 17:01:29 +00:00
tersec cd7da00d16
eliminate fcU/getPayload race condition causing missed proposals (#4800) 2023-04-12 12:33:21 +03:00
Etan Kissling 56986c08d6
rename `execution(Block|Payload)Root` > `executionBlockHash` (#4809)
Note: `execution_payload_root` is _actually_ `htr(payload)`.
Only `executionPayloadRoot` was used as `executionBlockHash`.
2023-04-11 23:31:47 +00:00
Etan Kissling dc22244b3e
fix `/eth/v1/debug/fork_choice` output (#4810)
Two fixes to `/eth/v1/debug/fork_choice`:

- `validity` enum is expected to be serialized as string instead of int
- `data` wrapper is not expected for this endpoint
2023-04-11 23:17:48 +00:00
Etan Kissling c3d043c0e1
rename `loadExecutionBlockRoot` > `loadExecutionBlockHash` (#4807)
There are still some `executionBlockRoot` after this, separate rename.
2023-04-11 16:56:29 +00:00
tersec eed34e740a
implement the getStateRandao Beacon API (#4799) 2023-04-11 18:27:48 +03:00
tersec 0fbf911722
use higher of available engine and builder API bids (#4795) 2023-04-11 18:19:48 +03:00
henridf a543b0b446
Quarantine.cleanupOrphans(): Delete unviable orphan (#4782) 2023-04-11 08:51:01 +02:00
Etan Kissling df7ecd4fe9
update `/eth/v1/debug/fork_choice` format to latest (#4802)
Syncs the `/eth/v1/debug/fork_choice` REST endpoint with latest specs.

- Validity is now reported as tri-state `enum` instead of two `bool`s
- Response includes store's justified and finalized checkpoints
- Additional `ExtraData` field on outer layer (empty for now)

https://github.com/ethereum/beacon-APIs/pull/232
2023-04-10 15:35:14 +00:00
Etan Kissling 450f06566b
accelerate execution layer sync using light client (#4805)
Turn on `--sync-light-client` option by default, now that it has shown
stability in local testnets.
2023-04-10 14:28:46 +00:00
Etan Kissling 566c178e59
add ANSI art for Deneb 🐟 (#4803)
Trigger ANSI art on upgrade to Deneb

Art created by http://beatscribe.com (beatscribe#1008 on Discord)
2023-04-10 12:42:19 +00:00
Etan Kissling b7d08d0a38
do not report pre-Merge sync progress as `/opt` (#4801)
Before the merge, assume `payloadStatus == NewPaylodStatus.valid` to
avoid cases of sync progress being reported with `/opt` suffix.
2023-04-09 14:58:20 +00:00
tersec 7857c47a8f
fix expected withdrawal mismatch logging (#4798) 2023-04-08 09:15:09 +00:00
tersec 742de1347d
only add active validators to knownValidators at startup (#4793) 2023-04-06 13:53:06 +00:00
Eugene Kabanov 0ff86e9538
web3signer refactoring and test suite. (#4775)
* Refactor nimbus_signing_node to support Unix signals.

* Fix SN unable to close REST server properly.

* Fix `keys`, `deposit` and `validator_registration` endpoints issues.
Add getValidatorExitSignature() and getDepositMessageSignature() to validator_pool.

* Add /reload endpoint and implementation.
Fix signData to not cancel `timer`.
Fix validator_pool should clear attachedValidators table.

* Diva protocol enhancement implementation.
2023-04-06 16:16:21 +03:00
Zahary Karadjov d5f454a8f5
Fix a logical typo 2023-04-06 12:53:44 +03:00
zah b59f9f5e1a
Remove some doAsserts that are easy to trigger from user actions (#4791) 2023-04-05 18:52:42 +00:00
tersec 3bfff6f219
query EL and builder relay for bids in parallel (#4749)
* outline for comparing bids from builder and engine API in BN

* set up async

* decision scaffold

* clean up logging

* Refactor proposeBlockMEV

* Update beacon_chain/validators/validator_duties.nim

Co-authored-by: zah <zahary@status.im>

* Update beacon_chain/validators/validator_duties.nim

Co-authored-by: zah <zahary@status.im>

* use typedescs instead of explicit generic parameters

---------

Co-authored-by: zah <zahary@status.im>
2023-04-05 16:35:32 +03:00
henridf 04302081b4
Add blob sidecar gossip validation (#4785)
* Add blob gossip validation

* lint

* Add test for getBlobSidecarTopic

* Fix closure variable capture issue

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-04-04 15:12:34 +00:00
Tanguy 5167a373ab
Bump libp2p (#4786)
* auto-bump nim-libp2p

* remove iWantBudget

---------

Co-authored-by: = <Menduist@users.noreply.github.com>
2023-04-04 13:31:41 +00:00
Eugene Kabanov e2bf58a6f2
Fix issue when VC crashes on invalid `beacon-node` argument. (#4765)
Make `beacon-node` URL argument less strict and more UX friendly.
2023-04-03 16:25:16 +00:00
tersec 464e680efc
detect missing network definitions at compile-time (#4784) 2023-04-03 15:39:12 +00:00
Eugene Kabanov 9e5c19eba5
Adjust remote signer idle connection timeout defaults. (#4779)
* Adjust remote signer idle connection timeout defaults.

* Address review comments.
2023-04-02 10:38:28 +00:00
tersec c31fbc3977
remove most std/options imports (#4778) 2023-03-31 20:46:47 +00:00
tersec b4508b853a
assorted consensus spec URL updates (#4776) 2023-03-30 23:08:52 +00:00
Zahary Karadjov 499b870a1a
Allow the git revision to be specified with -d:git_revision_override=xxxxxx 2023-03-30 20:01:20 +03:00
tersec f71a279d17
more deneb support in ncli_db and forks (#4774) 2023-03-30 10:06:23 +00:00
tersec 0d051b3da8
add Deneb support to REST (de)serialization (#4770) 2023-03-29 11:41:54 +03:00
tersec aa04d45335
use canonical, standalone sepolia network definition repository (#4762) 2023-03-29 11:37:14 +03:00
tersec 9d2fdc2e26
assorted consensus spec URL updates to v1.3.0-rc.5 (#4768) 2023-03-27 14:41:22 +00:00
tersec 29c77948cb
document rationale for await/awaitWithTimeout selection (#4764) 2023-03-23 19:54:41 +00:00
tersec 69bc9bcd85
no timeout for purely local async slashing protection and signing (#4751) 2023-03-23 16:51:55 +02:00
henridf 635a924e8c
Add nim-kzg4844 and use it in validate_blobs (#4732) 2023-03-23 09:47:04 +00:00
tersec fc1f9a2065
builder API liveness failsafe (#4746)
* builder API liveness failsafe

* add test summary change
2023-03-22 18:48:48 +01:00
Zahary Karadjov 6c0d756d54
Version 23.3.2 2023-03-22 01:50:47 +02:00
tersec 44b38ef137
Implement /eth/v1/validator/{beacon,sync}_committee_selections Beacon API end-points (#4760) 2023-03-22 01:47:36 +02:00
zah eb678be0ff
Don't report an 'EL connection degraded' warning when the EL is not synced (#4761) 2023-03-22 00:19:36 +02:00
zah b8f2022668
Remove an overly agressive assertion (#4759) 2023-03-21 23:35:22 +02:00
Zahary Karadjov 813fddee8c
Capella hard-fork epoch for Mainnet 2023-03-21 23:33:06 +02:00
Zahary Karadjov 865c84cc18
Some documentation for the new 'deposit exit' work-around 2023-03-21 23:06:57 +02:00
cheatfate 596006be08
Workaround for issue #4216. 2023-03-21 23:06:57 +02:00
Tanguy 769ed00203
Add light client gossip metrics (#4745) 2023-03-21 08:55:48 +01:00
tersec 2f634c10a4
automated consensus spec URL updating from v1.3.0-rc.4 to rc.5 (#4756) 2023-03-21 00:42:22 +00:00
tersec 82bf949561
use consensus spec v1.3.0-rc.5 test vectors (#4754) 2023-03-20 20:17:25 +00:00
Zahary Karadjov 4d1b2dd9f5
Merge branch 'stable' into unstable 2023-03-17 17:51:39 +02:00
tersec 0c62950de6
more consensus spec v1.3.0-rc.4 URL updates (#4744) 2023-03-17 12:14:08 +00:00
tersec ec77116414
automated consensus spec URL updating from v1.3.0-rc.3 to rc.4 (#4742) 2023-03-17 01:10:31 +00:00
Etan Kissling fb26e19de9
use v1.3.0-rc.4 consensus spec test vectors (#4740) 2023-03-16 21:35:20 +01:00
Zahary Karadjov 3615606914
Version 23.3.1 2023-03-14 19:43:11 +02:00
Zahary Karadjov ee610cbf34
Restore the hidden --web3-force-polling option as a deprecated one 2023-03-14 18:54:15 +02:00
Zahary Karadjov 957a8c2fdc
Bugfix: A single unsynced EL client may cause an inappropriate switch to optimistic mode. 2023-03-14 18:19:11 +02:00
Zahary Karadjov d899a6a834
Prevent a potential panic on dereferencing a 'none' value 2023-03-14 18:05:29 +02:00
Zahary Karadjov 46f48269ef
Backwards compatible handling of the web3-url parameter in TOML 2023-03-14 17:50:03 +02:00
Zahary Karadjov 3a35809a02
Backwards-compatible handling of Engine URLs that don't include a specified protocol 2023-03-14 14:02:50 +02:00
henridf 9906a0d5a2
Gate Deneb `beacon_block` subscription (#4735) 2023-03-14 11:52:01 +00:00
Zahary Karadjov 2ee15a38ed
Reduce the number of block timestamp requests
The timestamps are only relevant within the candidate block range.
2023-03-14 13:47:13 +02:00
Zahary Karadjov 6fb48aca7d
Lower the maximum block range in getLogs requests to 1000 2023-03-14 13:45:01 +02:00
Etan Kissling ceefc49965
rm redundant `ConsensusFork.toString` (#4733) 2023-03-14 01:24:10 +00:00
Etan Kissling 5ef1ed5d96
indicate Capella / Shanghai readiness on startup (#4630)
In startup "Launching beacon node" message, log readiness for Capella /
Shanghai as part of config to aid with support requests.
2023-03-13 22:01:15 +01:00
Etan Kissling 29a699f69d
rename `blockFork` > `consensusFork` (#4731)
* rename `blockFork` > `consensusFork`

* revert one overzealous stateFork / blockFork rename
2023-03-12 18:48:38 +00:00
Etan Kissling f0dcbaacff
extend `addTestBlock` with Capella/Deneb support (#4728)
* extend `addTestBlock` with Capella/Deneb support

* bump copyright years

* fix
2023-03-11 20:11:33 +00:00
Etan Kissling fd8e86972d
rename `lcDataForkAtStateFork` > `lcDataForkAtConsensusFork` (#4726) 2023-03-11 20:09:21 +00:00
Etan Kissling 3e872bec86
rename `eip4844_preset.nim` > `deneb_preset.nim` (#4724) 2023-03-11 17:00:35 +00:00
Etan Kissling d8f39dd24f
rename `stateForkForDigest` > `consensusForkForDigest` (#4727) 2023-03-11 16:58:48 +00:00
Etan Kissling b0cccae7f4
rename `(with|at)StateFork` > `(with|at)ConsensusFork` (#4725) 2023-03-11 14:39:29 +00:00
Etan Kissling eba5f9af26
rename `upgrade_to_eip4844` > `upgrade_to_deneb` (#4723) 2023-03-11 14:28:55 +00:00
Etan Kissling e542d079b0
rename Deneb database tables (not in testnets atm) (#4720) 2023-03-11 02:26:03 +01:00