5871 Commits

Author SHA1 Message Date
Etan Kissling
f98c33ad03
generalize commonAncestor function to BlockId (#5192)
To enable additional use cases, e.g., `/states/###/randao` beacon API,
`ShufflingRef` acceleration logic needs to be able to operate on parts
of the DAG that do not have `BlockRef`. Changing `commonAncestor` to
act on `BlockId` instead of `BlockRef` is a step toward that and also
simplifies the logic some more.
2023-07-18 17:37:53 +02:00
tersec
3a818ecb93
fall back to non-fcu fork choice on epoch boundaries (#5195)
* fall back to non-fcu fork choice on epoch boundaries

* Future[bool]

* fix

* Update beacon_chain/consensus_object_pools/consensus_manager.nim

Co-authored-by: Etan Kissling <etan@status.im>

* make things consistent with Opt[void] return

---------

Co-authored-by: Etan Kissling <etan@status.im>
2023-07-17 22:30:38 +02:00
Miran
92da61ce88
update the RPi guide regarding overclocking (#5135) 2023-07-17 10:26:38 +02:00
Etan Kissling
2efc44a8ab
accelerate RANDAO computation for post-merge blocks (#5190)
Post-merge blocks contain all information to directly obtain RANDAO
without having to load any additional info. Take advantage of that to
further accelerate `ShufflingRef` computation. Note that it is still
necessary to verify that `blck` / `state` share a sufficiently recent
ancestor for the purpose of computing attester shufflings.

- new: 243.71s, 239.67s, 237.32s, 238.36s, 239.57s
- old: 251.33s, 234.29s, 249.28s, 237.03s, 236.78s
2023-07-15 22:16:56 +02:00
tersec
565edfa351
remove unused code (#5158)
* remove unused code

* add copyright header
2023-07-15 16:30:52 +00:00
Etan Kissling
f1963bf599
allow initializing test DBs from later forks (#5186)
Running `makeTestDB` in tests currently always initializes DB with a
`phase0` state, preventing tests that configure a fork schedule that
starts in a different fork from working properly. Fix that by upgrading
the genesis state to whatever fork the fork schedule starts with.
2023-07-14 09:35:58 +02:00
tersec
703eead2e6
ensure the arrays in BeaconStateDiff are int-indexable in Nim 2.0 (#5189) 2023-07-13 23:25:10 +02:00
tersec
c549fe6a65
bump nim-json-rpc to 32200345f80a3b5f72e15d1b4e9363363d95a1cd (#5187) 2023-07-12 22:44:17 +02:00
tersec
0be5b5b8f2
handle distinct nil more rigorously (#5180) 2023-07-12 17:33:19 +00:00
Etan Kissling
74bb4b1411
simplify RANDAO recovery in ShufflingRef acceleration (#5183)
Current RANDAO recovery logic is quite complex as it optimizes for the
minimum amount of database reads. Loading blocks isn't the bottleneck
though, so rather make the implementation more concise by avoiding the
complex strategy planning step. Note that this also prepares for an even
faster implementation for post-merge blocks in the future that extracts
RANDAO from `ExecutionPayload` directly if available, so even in cases
where efficiency is slightly lower, only historical data is affected.

`time nim c -r tests/test_blockchain_dag` (cached binary):

- new: 145.45s, 133.59s, 144.65s, 127.69s, 136.14s
- old: 149.15s, 150.84s, 135.77s, 137.49s, 133.89s
2023-07-12 17:27:05 +02:00
Jacek Sieka
0660ffcd3e
fix debug logging 2023-07-11 21:48:27 +02:00
henridf
f13f23e21b
Complete Deneb bn side for /eth/v2/validator/blocks/{slot} (#5146)
* Complete Deneb bn side for /eth/v2/validator/blocks/{slot}

* Address review feedback

* More review feedback

* Tighten up when condition

* Update beacon_chain/rpc/rest_validator_api.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 19:35:55 +00:00
Jacek Sieka
a2adbf809f
Perform block pre-check before validating execution (#5169)
* Perform block pre-check before validating execution

When syncing, blocks have not been gossip-validated and are therefore
prone to trivial faults like being known-unviable, duplicate or missing
their parent.

In addition, the duplicate-block check in BlockProcessor was not
considering the quarantine flow and would therefore cause
recently-quarantined blocks to be silenty dropped when their parent
appears delaying the sync end-game and thus causing longer startup
resync time.

This PR verifies trivial conditions before performing execution
validation thus avoiding duplicates and missing parents alike.

It also ensures that the fast-sync EL mode is used for finalized blocks
even if the EL is timing out / slow to respond - this allows the CL to
complete its sync faster and switch to "normal" lock-step at the head of
the chain more quickly, thus also allowing the EL to access the latest
consensensus information earlier.

* oops

* remove unused constant
2023-07-11 18:55:51 +02:00
Jacek Sieka
ca1775f725
Fetch by-root request directly from quarantine (#5167)
When the requestmanager is busy fetching blocks, the queue might get
filled with multiple entries of the same root - since there is no
deduplication, requests containing the same root multiple times will be
sent out.

Also, because the items sit in the queue for a long time potentially,
the request might be stale by the time that the manager is ready with
the previous request.

This PR removes the queue and directly fetches the blocks to download
from the quarantine which solves both problems (the quarantine already
de-duplicates and is clean of stale information).

Removing the queue for blobs is left for a future PR.

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 18:22:02 +02:00
tersec
174c33e5fa
residual cleanup from https://github.com/status-im/nimbus-eth2/pull/5152 (#5181) 2023-07-11 14:36:37 +00:00
Jacek Sieka
185684daf8
simplify deposit syncing message (#5177)
and make it less alarming since there's no action required from the user
typically
2023-07-11 08:01:56 +02:00
Jacek Sieka
1bac029935
avoid unnecessary buffer zero:ing (#5178) 2023-07-11 08:01:45 +02:00
tersec
a1ddd447bc
use correct import name for kzg4844 (#5179) 2023-07-10 23:55:40 +00:00
Etan Kissling
5115aaedb7
early exit commonAncestor when comparing with finalizedHead (#5174)
* early exit `commonAncestor` when comparing with `finalizedHead`

As all `BlockRef` lead to `finalizedHead` (`parent == nil`),
can shortcut in that situation and immediately return `finalizedHead`
if passed as one of the arguments.

* typo in comment

* add test from #5152

Co-authored-by: tersec <tersec@users.noreply.github.com>

* add note about test complexity

* regenerate test summary

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-10 20:36:25 +00:00
Etan Kissling
7fc99ff040
bump gnosis-chain-configs to 9f01035125054f16022929116dfbd0190f7bafd7 (#5175)
- Adding new bootnodes
2023-07-10 15:46:57 +00:00
henridf
86e33b0a3f
Remove stale denebImplementationMissing (#5173)
This should have been removed as part of #5137
2023-07-10 17:17:33 +02:00
henridf
23edc4f960
Finish readValue for deneb RestPublishedBeaconBlockBody (#5172)
(Stray piece that got left behind a previous PR...)
2023-07-10 14:57:11 +00:00
dependabot[bot]
b3a033c66f
Bump scipy from 1.7.3 to 1.10.0 in /ncli (#5166)
Bumps [scipy](https://github.com/scipy/scipy) from 1.7.3 to 1.10.0.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](https://github.com/scipy/scipy/compare/v1.7.3...v1.10.0)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-09 07:42:00 +00:00
54e1a3655e
ci: bump Jenkins maxConcurrentTotal to 9
We have added more powerful hosts with more slots(3 instead of 2)
which should be able to accomodate more Nimbus CI jobs.

https://github.com/status-im/infra-ci/issues/85

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 11:50:40 +02:00
Miran
f2785e3339
add a trusted node sync step to the guide (#5161) 2023-07-07 11:02:58 +02:00
tersec
49056c3e39
use v1.3.0 stability subnet behavior when appropriate (#5156) 2023-07-07 04:16:12 +00:00
Jacek Sieka
862897124a
rest: allow state root lookups up to 8k slots from head (#5155)
* rest: allow state root lookups up to 8k slots from head

* avoid int, clean up error messages
2023-07-06 12:53:35 +02:00
Jacek Sieka
b2c421f5d6
eth: bump (#5165)
> sqlite3: better error message
  > Use results Opt for discv5 getNode and resolve
  > implement EIP-4844: Shard Blobs Transactions
  > Enable tests with also json sink and fix failures
  > Style fixes and comment improvements on uTP code
  > fixes related to stint breaking changes
  > Fix potentential uTP socket leak for socket in SynSent state
  > Avoid double socket destroy on timed-out uTP SYN-ACK
  > remove stddefects import
  > use correct exception in `parseCmdArg`
2023-07-06 10:20:06 +00:00
Jacek Sieka
fa481dcb14
small bumps (#5164)
1.2/1.4 removals and cleanups
2023-07-06 10:16:27 +00:00
Eugene Kabanov
ac1b02698a
VC: Use scoring function to select best attestation data when using multiple BNs. (#5101)
* Initial commit.

* Move score selection log statement to debug level.

* Fix proper float64 log format.

* Cleanup imports and legacy code.

* Address review comments.

* Address review comments.

* Fix scoring function.

* Address review comments.

* Address review comments 2.
Fix registerBlock post-rebase issues.

* Simplify innerLoop decision making.

* Make getAttestationDataScore() more testable.
Add tests for getAttestationDataScore().

* Add modified AllTests copy.
2023-07-06 11:14:22 +02:00
Jacek Sieka
1e227b2704
stew without results (#4862)
https://github.com/status-im/nim-stew/pull/185

also an assortment of other stew fixes
2023-07-05 16:06:45 +00:00
tersec
1e2d3b12c4
update some consensus-specs ref URLs to v1.4.0-beta.0 (#5163) 2023-07-05 16:02:55 +00:00
Miran
038c97fdf3
various small docs improvements (#5059) 2023-07-05 15:43:43 +02:00
tersec
6730e16439
replace integer cast with conversion (#5160) 2023-07-04 14:29:26 +00:00
tersec
ad3b2fb5ed
correct execution client link in docs (#5159) 2023-07-04 10:54:32 +00:00
tersec
583eb4db17
work around some CastSizes warnings and fix unreachable code warnings (#5149) 2023-06-30 21:34:04 +02:00
tersec
0be95571a7
update some consensus spec URLs to v1.4.0-beta.0 (#5150) 2023-06-30 16:12:54 +00:00
tersec
b4c4f0955e
https://github.com/ethereum/consensus-specs/pull/3421 https://github.com/ethereum/execution-apis/pull/420 (#5147) 2023-06-30 08:14:20 +00:00
henridf
99264d7507
Local validator proposals (#5137)
* Implement local validator block proposals

* Address review feedback

* Fix rebase issue
2023-06-30 07:39:41 +00:00
tersec
ba94dc849f
opt-out support for v1.4.0 attestation stability subnets (#5128)
* opt-out support for v1.4.0 attestation stability subnets

* document debug-prefix-flag-policy and remove temporary from flag name
2023-06-30 05:23:15 +00:00
Etan Kissling
e8f3bfdb08
bump nimbus-build-system to fe9bc3f3759ae1add6bf8c899db2e75327f03782 (#5132)
- switch to Nim `v1.6.14`
2023-06-29 16:06:14 +00:00
tersec
66febf2444
bump nim-blscurve to b71a16312699f6d26dee6a710b4aafea6ee5680d (#5143) 2023-06-29 11:13:15 +00:00
tersec
d5a6d51ede
Web3Signer support only for only Bellatrix and later (#5107)
Co-authored-by: Etan Kissling <etan@status.im>
2023-06-29 09:36:43 +00:00
tersec
dc5687093f
update gossip validation for v1.4.0-beta.0 (#5133)
Co-authored-by: Etan Kissling <etan@status.im>
2023-06-29 08:34:21 +00:00
henridf
7b0e3b1c04
Review feedbac from #5106 (#5141) 2023-06-29 01:17:54 +00:00
Etan Kissling
3c3a4ff3f3
run tests against both submodule and latest Nim (#5140)
Currently CI only tests against status `version-1-6` branch.
Update to test against the selected commit through submodule lock,
as well as the latest upstream `version-1-6` instead.
2023-06-28 23:24:23 +02:00
henridf
1234900065
Add blob handling to message router (#5106)
* Add blob handling to message router

* address review feedback

* Fix typos
2023-06-28 17:55:31 +00:00
Etan Kissling
41b93ae57a
fix code for Nim v1.6.14 compatibility (#5131)
With `v1.6.14` there is compilation issue in `trusted_node_sync` where
a type is not inferred automatically anymore for a `nil` instance.
Fix it so we can bump the compiler.

See https://github.com/status-im/nimbus-build-system/pull/63
2023-06-28 17:21:43 +00:00
Eugene Kabanov
5e1a0eac85
VC: Runtime config (#5130)
* Make VC able to understand any type of `/eth/v1/config/spec` response without any changes in source code.
Update compatibility checking.
Now VC is able to obtain any constant from `spec` call.

* Remove RestSpecVC declaration.
2023-06-28 15:33:38 +02:00
Eugene Kabanov
c534a285b9
Fix polling block monitor from deadlocks when connection has been lost. (#5109)
Fix events block monitor from losing connection forever.
Add node to block received log statement.
2023-06-28 15:33:21 +02:00