5809 Commits

Author SHA1 Message Date
tersec
614202e30d
automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
Jacek Sieka
d9bff54d57
don't check legacy tables for pruning (#5116)
These tables can't be deleted from (read-only) and would be too slow to
delete from anyway due to the inefficient storage format in use.

* slow down startup clearing too
* remove unused del function
2023-06-24 16:14:28 +02:00
tersec
9dbf63d8ca
use v1.4.0-beta.0 consensus spec test vectors (#5120) 2023-06-24 12:27:15 +00:00
tersec
b62664915e
consensus spec v1.4.0 attestation stability subnets (#5092) 2023-06-23 09:30:46 +00:00
henridf
fa212515f5
Implement validor client proposals for Deneb (#5094) 2023-06-23 09:30:16 +00:00
Tanguy
11d18acfeb
Add Yamux experimental support (#5080) 2023-06-23 09:16:30 +02:00
Etan Kissling
8a853fc4e4
extract LC sync task scheduling helper function (#5117)
The helper function to compute delay until next light client sync task
can be useful from more general purpose contexts. Move to helpers, and
change it to return `Duration` instead of `BeaconTime` for flexibility.
2023-06-23 06:19:51 +00:00
Zahary Karadjov
1bc9f3a67a
Disable the 32-bit ARM packaging v23.6.0 2023-06-22 11:19:16 +03:00
Etan Kissling
cbc4cef981
bump gnosis-chain-configs to 19060cb894ea0fb8ff3c8ec2fd26c0433f56c52c (#5114)
https://github.com/gnosischain/configs/pull/12

- Schedule shapella for gnosis
2023-06-22 11:14:02 +03:00
Etan Kissling
bab6a77abc
optimize LC sync task scheduling (#5112)
- When syncing `LightClientUpdatesByRange`, and peer replies with
  fewer periods than requested, no need to delay next request.
- When `FinalityUpdate` / `OptimisticUpdate` sync fails,
  no need to retry immediately.
2023-06-21 17:44:49 +00:00
Etan Kissling
6175fa8a79
bump nimbus-build-system to 239c3a7fbb88fd241da0ade3246fd2e5fcff4f25 (#5111)
- Use upstream build system when possible
- Fix checking out of recent Nim tags in shallow repo
- install Nimble CA cert when `skipIntegrityCheck`
2023-06-21 17:00:51 +02:00
Etan Kissling
6ec6b99ef6
extract nextLightClientSyncTask helper (#5108)
The `nextLightClientSyncTask` helper is generic enough to be useful for
contexts besides libp2p based light client sync. Extract it.
2023-06-21 06:28:14 +00:00
Etan Kissling
d670af5bf5
align style when calling LC libp2p API (#5104)
The `UpdatesByRange` API takes `startPeriod / count`, but is internally
called by `Slice`. Move the logic that converts from the `Slice` to the
caller to reduce complexity inside the used `doRequest` function.
2023-06-20 20:21:32 +00:00
Eugene Kabanov
e2f7065432
Fix iteration issue. (#5105)
Add elapsed_time to log statement.
2023-06-20 16:41:20 +00:00
Tanguy
d00788ab29
Bump libp2p to fix incoming connections (#5102) 2023-06-20 17:40:47 +02:00
Miran
cabb2779b4
Make the validating guide self-contained (#5076)
* make the validating guide self-contained

Instead of having links to the three separate how-to guides,
now everything is under one umbrella.

* fix all now-broken links

* Apply suggestions from code review

Co-authored-by: tersec <tersec@users.noreply.github.com>

* address review comments

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-06-20 14:38:10 +00:00
tersec
bb63f8e0ec
update docs to reflect default pruning change (#5103) 2023-06-20 14:25:13 +00:00
Zahary Karadjov
883468eb22
Update the guide to match the new default for the --history option 2023-06-20 12:59:36 +03:00
Zahary Karadjov
6b683924ce
Version 23.6.0 2023-06-20 02:11:44 +03:00
Zahary Karadjov
b32fac0a25
Merge branch 'stable' into unstable 2023-06-20 02:09:05 +03:00
Zahary Karadjov
503cde366c
Switch to --history:prune by default 2023-06-20 02:08:48 +03:00
Etan Kissling
2722778ce5
reduce nim-eth dependencies just for RNG (#5099)
We have several modules that import `nim-eth` for the sole purpose of
its `keys.newRng` function. This function is meanwhile a simple wrapper
around `nim-bearssl`'s `HmacDrbgContext.new()`, so the import doesn't
really serve a use anymore. Replace `keys.newRng` with the direct call
to reduce `nim-eth` imports.
2023-06-19 22:43:50 +00:00
Etan Kissling
3dd256b686
bump nim-zlib to f34ca261efd90f118dc1647beefd2f7a69b05d93 (#5096)
* bump `nim-zlib` to `f34ca261efd90f118dc1647beefd2f7a69b05d93`

- ignore *.exe
- nim 1.6 adjustment
- update github action to v3
- use Nim 2.0 in CI
- test both refc and ORC in post-1.6 Nim versions
- remove Nim 1.2 and 1.4 support

* bump `nim-websock` for compatibility

`nim-websock` `fea05cde8b123b38d1a0a8524b77efbc84daa848` is needed

- reduce compiler warnings
2023-06-19 18:47:11 +00:00
Jacek Sieka
4ae1857898
fix missing era regeneration (#5098)
+1 is already done via a `defer`
2023-06-19 15:47:24 +00:00
Etan Kissling
a92b249892
bump nim-websock to 4c5e225eeb342a3b9cfb2fbdddd92d00568b5553 (#5097)
- Remove newRng
- Fix closing hangs
- Remove recvMsg pre-allocation
2023-06-19 13:11:30 +00:00
henridf
0f8866d672
Add Deneb beacon API types (#5060)
* Add Deneb beacon API types

- Introduce the `ProduceBlockResponseV2` type for handling responses to `GET /eth/v2/validator/blocks/{slot}` (https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Validator/produceBlockV2)

- Introduce the `RestPublishedSignedBlockContents` type for handling the request body to `POST /eth/v1/beacon/blocks` (https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/ValidatorRequiredApi/publishBlock)

And a few other small changes towards Deneb block production support.

* Address review feedback

* Post-rebase fixes
2023-06-19 08:56:52 +00:00
Etan Kissling
4da9517f25
bump nim-serialization to 384eb2561ee755446cff512a8e057325848b86a7 (#5086)
- readFieldIMPL workaround when orc enabled
- remove nimble.lock
- remove nim 1.2 and nim 1.4 from ci
- remove Defect raises from makeFieldReadersTable
- another attempt to reduce compiler warnings
- More fix to exception tracking and generic_suite
- bump version to 0.2.0
- avoid implicit synthesis of `CaseTransition`
- make nimble file consistent with test matrix
- workaround for `NimYAML` issue
2023-06-17 08:41:01 +02:00
Etan Kissling
3a6b50cf21
require sync committee supermajority in CI (#5085)
* require sync committee supermajority in CI

To better catch problems with sync committee messages in CI, extend
local testnet simulation to also verify that each block is signed
by a supermajority of the sync committee.

Requires #5083 and #5084

* lint
2023-06-17 06:00:01 +00:00
Etan Kissling
8c6c8a0ffa
use correct slot when producing sync aggregate around forks (#5089)
`produceSyncAggregate` is called in new slot when block is produced,
while the other functions in `sync_committee_msg_pool` are called in
previous slot. So, need to subtract 1 slot when producing sync aggregate
to accept the signatures using the old digest during fork transition.
2023-06-16 21:30:36 +00:00
Etan Kissling
aa54760391
avoid implicit CaseTransition in keystore Json parsing (#5061)
Override default Json parsing for keystore case objects to avoid
`CaseTransition` logic to be emitted. When parsing the discriminator,
reinitialize the entire object instead of implicitly changing it,
to avoid UB and also possible oversights when extending the object.
See https://github.com/status-im/nim-serialization/pull/59
2023-06-16 19:02:53 +02:00
tersec
591c2246d5
update consensus spec URLs to v1.4.0-alpha.3 (#5088) 2023-06-16 16:45:09 +00:00
henridf
ae29babd9f
Implement blob sidecar signatures (#5072)
* Implement blob sidecar signatures

* Update allTests
2023-06-16 15:47:18 +00:00
Etan Kissling
8eec6ab221
avoid ProveField warning in network_metadata (#5066)
Before assigning to `genesisData` or returning `cfg`, have to check that
metadata is not `incompatible` to avoid `ProveField` warning.
The way how we use it was already correct (`incompatible` unreachable).
Use `case` syntax to silence the warning, and add comments referring to
the existing checks that make `incompatible` unreachable.
2023-06-16 12:15:42 +00:00
Etan Kissling
e9262ab6a3
ensure sync duties for next epoch are registered in time (#5084)
* ensure sync duties for next epoch are registered in time

For attestations, VC queries duties for current and next epoch.
For sync messages, VC queries for current and next period (if soon).
This means that for sync messages we don't actually have the duties for
next epoch in all situations, leading to situation where VC may miss
sync duties in the final slot of an epoch when using. As duties remain
same within a sync committee period, simply copy them over to next epoch
to avoid this situation without adding network latency.

* Update beacon_chain/validator_client/duties_service.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-06-16 08:42:57 +00:00
Jacob Shufro
11f5250ce4
Lower ClientMaximumValidatorIds to match beacon spec limit (#5082) 2023-06-16 09:01:16 +02:00
Etan Kissling
6f68c1a46e
query sync duties for correct slot (#5083)
VC currently misses sync committee duties for first slot of most epochs
because the 1 slot offset is not taken into account. Duties for the next
slot must be used during any given current slot. We use the correct slot
for processing the duty, but do not use the correct slot for fetching.
2023-06-16 06:33:13 +00:00
Etan Kissling
a6f0a7a55d
broaden SszError catches to SerializationError (#5081)
`nim-serialization` is tagged with `{.raises:[SerializationError].}` so
it is no longer sufficient to catch `SszError` in some situations.
`SszError` inherits from `SerializationError`, so broadening the caught
exception types can be done now, to enable bumping `nim-serialization`.

https://github.com/status-im/nimbus-eth2/pull/5043#issuecomment-1584227993

#5061 is also needed to bump `nim-serialization`.
2023-06-15 16:29:13 +00:00
Tanguy
98ab0afcc3
Bump libp2p (#5079)
* Bump libp2p

* Switch to unstable
2023-06-15 12:59:35 +00:00
Etan Kissling
142a3e91e5
ProveField cleanups in keystore (#5050)
Cleanup for `ProveField` warnings in `keystore` module.
Note that `ProveField` is disabled by default in makefile, but sometimes
these pop up when doing a regular `nim c`, and cleaning these may allow
enabling the warning in some future.
2023-06-15 14:53:42 +02:00
tersec
ce1503b51f
remove web3signer block signing V1 (phase0 only) support (#5014) 2023-06-15 14:52:19 +02:00
tersec
939b3002e4
remove removed getDebugChainHeadsV1; fix API links in documentations (#5058) 2023-06-15 09:58:38 +00:00
Etan Kissling
411d809c82
split file loading from parsing in helpers (#5054)
* split file loading from parsing in helpers

In `readSszForkedHashedBeaconState` and `readRuntimeConfig`, split the
part that loads the file from the part that parses the file. The parsing
portion can be reused with that, e.g., when loading from the network.

* add missing export marker
2023-06-14 19:48:50 +00:00
Etan Kissling
114453e67d
remove SAFE_SLOTS_TO_UPDATE_JUSTIFIED (#5078)
The `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` constant is no longer used in any
EF or Gnosis network metadata definitions. We can remove it.

Upstream changes:

- https://github.com/eth-clients/eth2-networks/pull/87
- https://github.com/gnosischain/configs/pull/10

Which we updated here:

- https://github.com/status-im/nimbus-eth2/pull/5071
- https://github.com/status-im/nimbus-eth2/pull/5077
2023-06-14 18:18:53 +00:00
Etan Kissling
aab26379ff
bump eth2-networks to e930d81f7c9db816c88d1a9336be8cef858f7f4d (#5077)
- remove `SAFE_SLOTS_TO_UPDATE_JUSTIFIED`
2023-06-14 18:18:02 +00:00
Etan Kissling
088c7e953d
bump gnosis-chain-configs to 41ffb72f67215b2f0c844498071844f3286c7fbb (#5071)
- remove `SAFE_SLOTS_TO_UPDATE_JUSTIFIED`
2023-06-14 14:47:29 +00:00
tersec
c27a65129b
add ability to load KZG trusted setup via runtime flag (#5075) 2023-06-14 10:52:00 +02:00
Eugene Kabanov
927180f36f
VC+BN: Validator voluntary exits through the Keymanager API (#5020)
* Initial commit.

* Address review comments.
2023-06-14 08:46:01 +02:00
Eugene Kabanov
c0e5c26da1
VC: Use SSZ encoding while processing blocks data (#4999)
* Refactor api.nim to provide more informative failure reasons.
Distinct between unexpected data and unexpected code.
Deprecate Option[T] usage.

* Fix 400 for produceBlindedBlock().
Get proper string conversion for strategy.

* Fix SSZ encoded versions of ProduceBlockResponseV2, ProduceBlockResponseV2 can be received and decoded.
Fix done() warnings.
Bump presto.

* Fix compilation error with new presto.
Use TcpNoDelay option for Web3Signer.

* Fix produceBlockV2() should provide SSZ responses too.

* Address block encoding issue.

* Fix signing test.

* Bump presto.

* Address review comments.
2023-06-14 08:04:15 +02:00
Etan Kissling
845bd3d570
ensure that submodule bumps are reachable (#5070)
It occurs sometimes that a submodule is bumped to a PR commit instead of
the corresponding canonical branch (as registered in `.gitmodules`).
Because we typically use `squash`, that PR commit can subsequently
become unreachable, randomly breaking the build of `nimbus-eth2`.
Prevent these accidents by only allowing submodule bumps to commits
on the branch registered in `.gitmodules`. On private branches, simply
update `.gitmodules` to match the personal dev branch.
2023-06-14 01:34:49 +00:00
Etan Kissling
6671965fd9
split SSZ and JSON parsing logic in LC handler (#5055)
Use separate functions per format when parsing LC data from REST.
This allows to process events from the eventstream more directly,
as they are always JSON not SSZ. And also makes the code cleaner.
2023-06-13 21:28:17 +00:00