Commit Graph

5369 Commits

Author SHA1 Message Date
Etan Kissling 964417d8ba
cache computed mock pubkeys in tests (#4869)
We have a few tests that access `MockPubKeys`. Adding a cache provides
a minor speedup, as repeated pubkey computations are avoided.

64 cache hits: EF - Capella - Unittests - Light client - Sync protocol
64 cache hits: EF - Altair - Unittests - Light client - Sync protocol
256 cache hits: EF - Capella - Unittests - Light client - Sync protocol
256 cache hits: EF - Altair - Unittests - Light client - Sync protocol
2297 cache hits: Attestation pool processing [Preset: mainnet]
512 cache hits: Beacon chain DB [Preset: mainnet]
32 cache hits: Block processor [Preset: mainnet]
1082 cache hits: Validator change pool testing suite
192 cache hits: Gossip validation  [Preset: mainnet]
32 cache hits: Gossip validation - Extra
192 cache hits: Spec helpers
32 cache hits: Light client processor [Preset: mainnet]
96 cache hits: Light client [Preset: mainnet]
192 cache hits: Beacon state [Preset: mainnet]
37 cache hits: state diff tests [Preset: mainnet]
160 cache hits: Block pool processing [Preset: mainnet]
32 cache hits: Block pool altair processing [Preset: mainnet]
96 cache hits: chain DAG finalization tests [Preset: mainnet]
32 cache hits: Old database versions [Preset: mainnet]
64 cache hits: Diverging hardforks
96 cache hits: Backfill
32 cache hits: Starting states
32 cache hits: Latest valid hash [Preset: mainnet]
32 cache hits: Pruning

5912 cache hits total per `make -j test`.
2023-04-27 15:04:56 +00:00
Etan Kissling 445ece1157
align `MockPrivKeys` with EF keys (#4866)
Back then, Milagro interop used offset 1000 for mock BLS keys.
Meanwhile, interop code was removed and multi client testnets are there.
EF tests use an offset of 1 for mock BLS keys. This patch aligns our
implementation to also use offset of 1, potentially making debugging of
state differences a bit easier (but, ultimately, low impact).

Furthermore, simulation files are now emitted into a subdirectory
to have less chunk in the repo root directory, and to avoid conflicts
where a cached file with offset 1000 runs against tests expecting 1.

See https://github.com/status-im/nimbus-eth2/pull/2928/files#r719266863
2023-04-27 12:17:19 +00:00
Etan Kissling dd1ffa5ded
add `el_offline` to `/eth/v1/node/syncing` (#4860)
Add compatibility with https://github.com/ethereum/beacon-APIs/pull/290
to the beacon node. Behaviour when configured with multiple ELs is not
specified; intention suggests to indicate whether all ELs are offline.
2023-04-27 10:47:38 +02:00
henridf ea8ef5799e
Fill in missing BlobSidecar gossip check (#4863) 2023-04-26 17:33:33 +00:00
Tanguy 6c3b9fdf36
Bump chronos (#4861) 2023-04-26 14:42:53 +00:00
Etan Kissling 107f733365
make `--trusted-block-root` option visible for CP sync (#4859)
Add `--trusted-block-root` to `--help` for `trustedNodeSync`.
2023-04-26 13:37:58 +02:00
Etan Kissling c4e258af2a
check signatures in CP sync backfill with LC (#4858)
When using trusted node sync with light client (`--trusted-block-root`),
the trust assumption on the server is reduced to solely be responsible
for data availability, but not data correctness. This means that we must
check block proposer signatures against the downloaded checkpoint, as
they are not covered by the block root.

Note that this lowers the backfill speed when using LC based CP sync
due to the extra checks, by about 60% for me.
2023-04-26 13:37:27 +02:00
tersec 1ccb36b272
include small dedup in block processor to handle blockByRoot blocks (#4850) 2023-04-26 07:00:03 +00:00
zah 55ae7d3b00
Version 23.4.0 (#4857)
* Draft release notes for version 23.4.0

* Drop the reference to the Builder API failsafe spec PR
2023-04-25 22:46:38 +03:00
henridf da6169bfe9
BlockProcessor.storeBlock: write blobs to DB (#4854) 2023-04-25 13:55:35 +03:00
zah 9b2c07c118
Allow exiting multiple validators at once (#4855) 2023-04-25 09:44:01 +03:00
Jacek Sieka 58b93ccbe0
era: Capella+ support (fixes #4752) (#4853)
Post-Capella, historical roots are computed from historical summaries
instead of being directly stored in the beacon state.

Slightly messy to pass both lists around - this is done to avoid
computing the historical root unnecessarily.
2023-04-24 15:26:28 +02:00
tersec fbe90dcbea
update statediffs to support capella (#4852) 2023-04-23 22:15:14 +03:00
tersec ed7ad56d16
allow for biasing comparisons between engine/builder api bids (#4847) 2023-04-23 22:10:34 +03:00
tersec b390911e93
automatic update of v1.3.0-rc.5 consensus spec URLs to v1.3.0 (#4848) 2023-04-21 18:52:43 +00:00
tersec 630ab3567e
actually reach capella fork epoch in mainnet local sims (#4845) 2023-04-20 18:18:35 +03:00
tersec c0b90833db
don't crash on block production error; build lookahead proposal from current head (#4842) 2023-04-20 15:07:27 +00:00
tersec d3400ca11b
low attestations during epoch should instafail in CI; dbg -> warn level in `newPayload` log (#4830)
* low attestations during epoch should instafail in CI; dbg -> warn level on newPayload log

* improve newPayload warning message when no valid EL connected

* reduce potential spam; make log spelling more consistent; use fatal/quit
2023-04-19 19:42:30 +00:00
tersec 2246a6ec95
Revert "include small dedup in block processor to handle blockByRoot blocks (#4814)" (#4840)
This reverts commit 8b3ffec0d5.

Syncing was broken with this: https://github.com/status-im/infra-nimbus/issues/132#issuecomment-1514465481
2023-04-19 19:16:27 +00:00
henridf 12d640b691
Request manager: Handle blobless blocks (#4833)
Post-Deneb, when the request manager receives a missing block from a
peer, it needs to check if the corresponding blobs are available, and
if so pass them along. If they aren't available, the newly-fetched
block must be put in blobless quarantine (while the blobs are
retrieved, coming in next commit).
2023-04-19 19:37:38 +03:00
tersec 0fc89484de
ignore EL exchangeTransitionConfiguration results (#4831) 2023-04-19 12:16:50 +00:00
tersec 4643a357ef
consensus spec ref URL update to v1.3.0-rc.5 (#4837) 2023-04-18 22:00:06 +00:00
tersec 45f89f1205
use consensus specs v1.3.0 test vectors (#4838) 2023-04-18 22:01:49 +02:00
Etan Kissling e6e4ba9de6
clean up redundant tests and config (#4836)
The consensus-spec-tests already cover the scenarios of our custom test
runner, so the custom tests can be removed. Also cleans up unused config
flags and related unreachable logic.
2023-04-18 21:26:36 +02:00
tersec 77c0aa713e
use non-nil head corresponding to execution head (#4835) 2023-04-18 14:17:56 +00:00
Etan Kissling 1d3e838236
lower log level of LC optimistic sync (#4834)
When using light client sync while main forward sync is behind, use
DEBUG log level instead of INFO log level to reduce log verbosity.
2023-04-18 13:37:29 +00:00
Zahary Karadjov 69d737e61d
Remove the Makefile targets related to the deprecated zhejiang testnet 2023-04-18 15:07:11 +03:00
tersec cc24429828
remove empty block fallback now that capella's on mainnnet (#4821)
* remove empty block fallback now that capella's on mainnnet

* build_empty_execution_payload is only testing infrastructure now

* update error message
2023-04-18 09:21:15 +00:00
henridf 8f2f40f0af
Remove blockBlobsVerifier (#4829)
Having a separate 'blockBlobsVerifier' function for post-Deneb blocks
is no longer of any benefit. This commit removes it.
2023-04-18 02:12:57 +02:00
Eugene Kabanov b51152153a
VC: Hardening and optimizing time handling. (#4743)
* Fix durationToNextSlot() and durationToNextEpoch() to work not only after Genesis, but also before Genesis.
Change VC pre-genesis behavior, add runPreGenesisWaitingLoop() and runGenesisWaitingLoop().
Add checkedWaitForSlot() and checkedWaitForNextSlot() to strictly check current time and print warnings.
Fix VC main loop to use checkedWaitForNextSlot().
Fix attestation_service to run attestations processing only until the end of the duty slot.
Change attestation_service main loop to use checkedWaitForNextSlot().
Change block_service to properly cancel all the pending proposer tasks.
Use checkedWaitForSlot to wait for block proposal.
Fix block_service waitForBlockPublished() to be compatible with BN.
Fix sync_committee_service to avoid asyncSpawn.
Fix sync_committee_service to run only until the end of the duty slot.
Fix sync_committee_service to use checkedWaitForNextSlot().

* Refactor validator logging.
Fix aggregated attestation publishing missing delay.

* Fix doppelganger detection should not start at pre-genesis time.
Fix fallback service sync status spam.
Fix false `sync committee subnets subscription error`.

* Address review comments part 1.

* Address review comments.

* Fix condition issue for near genesis waiting loop.

* Address review comments.

* Address review comments 2.
2023-04-17 21:31:54 +00:00
tersec 228e10f1d9
update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
tersec 8b3ffec0d5
include small dedup in block processor to handle blockByRoot blocks (#4814) 2023-04-17 19:36:15 +00:00
Jacek Sieka 7df75d77fa
move spec constants to spec (#4825)
* move spec constants to spec

quite funny, this removes >_1000_ transitive imports when building
`nimbus_validator_client`.

`genDepend`:

```sh
[arnetheduck@praeceps beacon_chain]$ diff -u
nimbus_validator_client.dot.pre nimbus_validator_client.dot | wc
   1167    3507  103822
```

* imports

* better constants

* oops

* too many blobs

* reorder spec constants in fork order

* cleanup
2023-04-17 17:18:54 +00:00
henridf 176c80a3d5
Fix incorrect naming and use of Quarantine.peekBlobless (#4826)
The 'peek' name was incorrect as it was actually removing from the
table. It was consequently used incorrectly in block processing: the
blobless block wasn't returned to the table when it should be.
2023-04-17 18:10:13 +02:00
tersec 75be7d267d
always use fcUV2 in shapella even for non-proposer fcUs (#4817)
* always use fcUV2 in shapella even for non-proposer fcUs

* avoid template/proc naming conflict with libp2p/signed_envelope.nim having a payload proc
2023-04-17 16:17:52 +02:00
Jacek Sieka addb7bda2f
Merge remote-tracking branch 'origin/stable' into unstable 2023-04-17 13:41:50 +02:00
henridf 4df851f4f5
Fix incorrect handling of blobless blocks (#4822)
We were passing along all blocks to local processing, including those
for which we were missing blobs. This commit fixes that.
2023-04-17 10:54:04 +00:00
henridf 29b431e312
Simplify block quarantine blobless (#4824)
* Simplify block quarantine blobless

The quarantine blobless table was initially keyed off of (Eth2Digest,
ValidatorSig). This was modelled off the orphan table. The presence of
the signature in the key is necessary for orphans, because we can't
verify the signature for an orphan. That is not the case for a
blobless block, where the signature can be verified.

So this PR changes the blobless block table to be keyed off a
Eth2Digest only. This simplifies the retrieval and handling of
blobless blocks.

* review feedback
2023-04-16 08:37:56 +00:00
liftlines 9bc28254cb
Update index.md
Updated the donation address as per announcement.
2023-04-16 17:00:26 +10:00
Etan Kissling cb9e0eed49
allow trusted node sync based on LC trusted block root (#4736)
* allow trusted node sync based on LC trusted block root

Extends `trustedNodeSync` with a new `--trusted-block-root` option that
allows initializing a light client. No `--state-id` must be provided.
The beacon node will then use this light client to obtain the latest
finalized state from the remote server in a trust-minimized fashion.
Note that the provided `--trusted-block-root` should be somewhat recent,
and that security precautions such as comparing the state root against
block explorers is still recommended.

* fix

* workaround for `valueOr` limitations

* reduce magic numbers

* digest len > context len for readability

* move `cstring` conversion to caller

* avoid abbreviations

* `return` codestyle
2023-04-16 06:07:07 +00:00
henridf 57623af36a
Remove unnecessary field derefs in BlockProcessor.storeBlock (#4823) 2023-04-16 01:25:17 +00:00
tersec f6062a2be1
compute correct withdrawals for lookahead proposal fcUs at epoch boundaries (#4820) 2023-04-15 21:48:24 +00:00
tersec ae45e11f8e
log when attempting to validate with exited validators (#4812) 2023-04-15 21:31:24 +00:00
tersec 5b0442e6e6
remove submodule containing deprecated zhejiang testnet definition (#4819) 2023-04-15 21:31:15 +00:00
henridf 974b1657b4
use nim-blscurve fork with upgraded blst (#4788) 2023-04-14 12:36:54 +00:00
henridf 021de18e06
Quarantine and reassembly of gossiped blobs and blocks (#4808) 2023-04-13 19:11:40 +00:00
Etan Kissling fa3655527c
add `formatIt` for `ForkedLightClientXyz` (#4816)
* add `formatIt` for `ForkedLightClientXyz`

Even though the `ForkyLightClientXyz` have `formatIt`, they do not apply
when logging `ForkedLightClientXyz`, leading to large logs at times.
Defining `formatIt` for `ForkedLightClientXyz` fixes this.

* exports not needed
2023-04-13 15:26:17 +00:00
Miran 0ec7df1042
fix spelling in conf.nim (#4813) 2023-04-12 17:01:29 +00:00
tersec cd7da00d16
eliminate fcU/getPayload race condition causing missed proposals (#4800) 2023-04-12 12:33:21 +03:00
Etan Kissling 56986c08d6
rename `execution(Block|Payload)Root` > `executionBlockHash` (#4809)
Note: `execution_payload_root` is _actually_ `htr(payload)`.
Only `executionPayloadRoot` was used as `executionBlockHash`.
2023-04-11 23:31:47 +00:00