5809 Commits

Author SHA1 Message Date
Jacek Sieka
a2adbf809f
Perform block pre-check before validating execution (#5169)
* Perform block pre-check before validating execution

When syncing, blocks have not been gossip-validated and are therefore
prone to trivial faults like being known-unviable, duplicate or missing
their parent.

In addition, the duplicate-block check in BlockProcessor was not
considering the quarantine flow and would therefore cause
recently-quarantined blocks to be silenty dropped when their parent
appears delaying the sync end-game and thus causing longer startup
resync time.

This PR verifies trivial conditions before performing execution
validation thus avoiding duplicates and missing parents alike.

It also ensures that the fast-sync EL mode is used for finalized blocks
even if the EL is timing out / slow to respond - this allows the CL to
complete its sync faster and switch to "normal" lock-step at the head of
the chain more quickly, thus also allowing the EL to access the latest
consensensus information earlier.

* oops

* remove unused constant
2023-07-11 18:55:51 +02:00
Jacek Sieka
ca1775f725
Fetch by-root request directly from quarantine (#5167)
When the requestmanager is busy fetching blocks, the queue might get
filled with multiple entries of the same root - since there is no
deduplication, requests containing the same root multiple times will be
sent out.

Also, because the items sit in the queue for a long time potentially,
the request might be stale by the time that the manager is ready with
the previous request.

This PR removes the queue and directly fetches the blocks to download
from the quarantine which solves both problems (the quarantine already
de-duplicates and is clean of stale information).

Removing the queue for blobs is left for a future PR.

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 18:22:02 +02:00
tersec
174c33e5fa
residual cleanup from https://github.com/status-im/nimbus-eth2/pull/5152 (#5181) 2023-07-11 14:36:37 +00:00
Jacek Sieka
185684daf8
simplify deposit syncing message (#5177)
and make it less alarming since there's no action required from the user
typically
2023-07-11 08:01:56 +02:00
Jacek Sieka
1bac029935
avoid unnecessary buffer zero:ing (#5178) 2023-07-11 08:01:45 +02:00
tersec
a1ddd447bc
use correct import name for kzg4844 (#5179) 2023-07-10 23:55:40 +00:00
Etan Kissling
5115aaedb7
early exit commonAncestor when comparing with finalizedHead (#5174)
* early exit `commonAncestor` when comparing with `finalizedHead`

As all `BlockRef` lead to `finalizedHead` (`parent == nil`),
can shortcut in that situation and immediately return `finalizedHead`
if passed as one of the arguments.

* typo in comment

* add test from #5152

Co-authored-by: tersec <tersec@users.noreply.github.com>

* add note about test complexity

* regenerate test summary

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-10 20:36:25 +00:00
Etan Kissling
7fc99ff040
bump gnosis-chain-configs to 9f01035125054f16022929116dfbd0190f7bafd7 (#5175)
- Adding new bootnodes
2023-07-10 15:46:57 +00:00
henridf
86e33b0a3f
Remove stale denebImplementationMissing (#5173)
This should have been removed as part of #5137
2023-07-10 17:17:33 +02:00
henridf
23edc4f960
Finish readValue for deneb RestPublishedBeaconBlockBody (#5172)
(Stray piece that got left behind a previous PR...)
2023-07-10 14:57:11 +00:00
dependabot[bot]
b3a033c66f
Bump scipy from 1.7.3 to 1.10.0 in /ncli (#5166)
Bumps [scipy](https://github.com/scipy/scipy) from 1.7.3 to 1.10.0.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](https://github.com/scipy/scipy/compare/v1.7.3...v1.10.0)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-09 07:42:00 +00:00
54e1a3655e
ci: bump Jenkins maxConcurrentTotal to 9
We have added more powerful hosts with more slots(3 instead of 2)
which should be able to accomodate more Nimbus CI jobs.

https://github.com/status-im/infra-ci/issues/85

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 11:50:40 +02:00
Miran
f2785e3339
add a trusted node sync step to the guide (#5161) 2023-07-07 11:02:58 +02:00
tersec
49056c3e39
use v1.3.0 stability subnet behavior when appropriate (#5156) 2023-07-07 04:16:12 +00:00
Jacek Sieka
862897124a
rest: allow state root lookups up to 8k slots from head (#5155)
* rest: allow state root lookups up to 8k slots from head

* avoid int, clean up error messages
2023-07-06 12:53:35 +02:00
Jacek Sieka
b2c421f5d6
eth: bump (#5165)
> sqlite3: better error message
  > Use results Opt for discv5 getNode and resolve
  > implement EIP-4844: Shard Blobs Transactions
  > Enable tests with also json sink and fix failures
  > Style fixes and comment improvements on uTP code
  > fixes related to stint breaking changes
  > Fix potentential uTP socket leak for socket in SynSent state
  > Avoid double socket destroy on timed-out uTP SYN-ACK
  > remove stddefects import
  > use correct exception in `parseCmdArg`
2023-07-06 10:20:06 +00:00
Jacek Sieka
fa481dcb14
small bumps (#5164)
1.2/1.4 removals and cleanups
2023-07-06 10:16:27 +00:00
Eugene Kabanov
ac1b02698a
VC: Use scoring function to select best attestation data when using multiple BNs. (#5101)
* Initial commit.

* Move score selection log statement to debug level.

* Fix proper float64 log format.

* Cleanup imports and legacy code.

* Address review comments.

* Address review comments.

* Fix scoring function.

* Address review comments.

* Address review comments 2.
Fix registerBlock post-rebase issues.

* Simplify innerLoop decision making.

* Make getAttestationDataScore() more testable.
Add tests for getAttestationDataScore().

* Add modified AllTests copy.
2023-07-06 11:14:22 +02:00
Jacek Sieka
1e227b2704
stew without results (#4862)
https://github.com/status-im/nim-stew/pull/185

also an assortment of other stew fixes
2023-07-05 16:06:45 +00:00
tersec
1e2d3b12c4
update some consensus-specs ref URLs to v1.4.0-beta.0 (#5163) 2023-07-05 16:02:55 +00:00
Miran
038c97fdf3
various small docs improvements (#5059) 2023-07-05 15:43:43 +02:00
tersec
6730e16439
replace integer cast with conversion (#5160) 2023-07-04 14:29:26 +00:00
tersec
ad3b2fb5ed
correct execution client link in docs (#5159) 2023-07-04 10:54:32 +00:00
tersec
583eb4db17
work around some CastSizes warnings and fix unreachable code warnings (#5149) 2023-06-30 21:34:04 +02:00
tersec
0be95571a7
update some consensus spec URLs to v1.4.0-beta.0 (#5150) 2023-06-30 16:12:54 +00:00
tersec
b4c4f0955e
https://github.com/ethereum/consensus-specs/pull/3421 https://github.com/ethereum/execution-apis/pull/420 (#5147) 2023-06-30 08:14:20 +00:00
henridf
99264d7507
Local validator proposals (#5137)
* Implement local validator block proposals

* Address review feedback

* Fix rebase issue
2023-06-30 07:39:41 +00:00
tersec
ba94dc849f
opt-out support for v1.4.0 attestation stability subnets (#5128)
* opt-out support for v1.4.0 attestation stability subnets

* document debug-prefix-flag-policy and remove temporary from flag name
2023-06-30 05:23:15 +00:00
Etan Kissling
e8f3bfdb08
bump nimbus-build-system to fe9bc3f3759ae1add6bf8c899db2e75327f03782 (#5132)
- switch to Nim `v1.6.14`
2023-06-29 16:06:14 +00:00
tersec
66febf2444
bump nim-blscurve to b71a16312699f6d26dee6a710b4aafea6ee5680d (#5143) 2023-06-29 11:13:15 +00:00
tersec
d5a6d51ede
Web3Signer support only for only Bellatrix and later (#5107)
Co-authored-by: Etan Kissling <etan@status.im>
2023-06-29 09:36:43 +00:00
tersec
dc5687093f
update gossip validation for v1.4.0-beta.0 (#5133)
Co-authored-by: Etan Kissling <etan@status.im>
2023-06-29 08:34:21 +00:00
henridf
7b0e3b1c04
Review feedbac from #5106 (#5141) 2023-06-29 01:17:54 +00:00
Etan Kissling
3c3a4ff3f3
run tests against both submodule and latest Nim (#5140)
Currently CI only tests against status `version-1-6` branch.
Update to test against the selected commit through submodule lock,
as well as the latest upstream `version-1-6` instead.
2023-06-28 23:24:23 +02:00
henridf
1234900065
Add blob handling to message router (#5106)
* Add blob handling to message router

* address review feedback

* Fix typos
2023-06-28 17:55:31 +00:00
Etan Kissling
41b93ae57a
fix code for Nim v1.6.14 compatibility (#5131)
With `v1.6.14` there is compilation issue in `trusted_node_sync` where
a type is not inferred automatically anymore for a `nil` instance.
Fix it so we can bump the compiler.

See https://github.com/status-im/nimbus-build-system/pull/63
2023-06-28 17:21:43 +00:00
Eugene Kabanov
5e1a0eac85
VC: Runtime config (#5130)
* Make VC able to understand any type of `/eth/v1/config/spec` response without any changes in source code.
Update compatibility checking.
Now VC is able to obtain any constant from `spec` call.

* Remove RestSpecVC declaration.
2023-06-28 15:33:38 +02:00
Eugene Kabanov
c534a285b9
Fix polling block monitor from deadlocks when connection has been lost. (#5109)
Fix events block monitor from losing connection forever.
Add node to block received log statement.
2023-06-28 15:33:21 +02:00
Eugene Kabanov
c2c5d80a4f
VC: Remote BN clock offset monitoring. (#4846)
* Initial commit.

* Add algorithm in comment.
Remove delays.
Fix logging statement issues.
Change update from epoch to slot.

* Obtain timestamp earlier.

* Add processing delays into algorithm.

* Fix time offset logging to produce integers instead of strings.

* Address review comments.

* Fix copyright year.
Fix updateStatus().

* Remove fields from Slot start log statement.
Fix issues when BN do not support Nimbus Extensions.
Rename metric name and type change.

* Add beacon role to disable time offset check manually.
2023-06-28 15:33:07 +02:00
tersec
bf575aac57
make OnFooBlockAdded callback types generic (#5134) 2023-06-28 13:30:49 +00:00
tersec
657f115210
bump nim-faststreams to 720fc5e5c8e428d9d0af618e1e27c44b42350309 (#5129) 2023-06-27 20:50:49 +02:00
Zahary Karadjov
970f5dfc4e
Merge branch 'stable' into unstable 2023-06-27 05:40:58 +03:00
Zahary Karadjov
187e1a0633
Version 23.6.1 v23.6.1 2023-06-27 03:58:14 +03:00
Zahary Karadjov
af3ef5a9b0
Re-enable the ARM packaging 2023-06-27 03:58:10 +03:00
Jacek Sieka
17e74c0c8b
don't check legacy tables for pruning (#5116)
These tables can't be deleted from (read-only) and would be too slow to
delete from anyway due to the inefficient storage format in use.

* slow down startup clearing too
* remove unused del function
2023-06-26 19:30:45 +03:00
henridf
ac3d8939a8
Add Deneb handling to state_transition.makeBeaconBlock() (#5118)
* Add Deneb handling to state_transition.makeBeaconBlock()

* review feedback
2023-06-26 16:05:02 +00:00
Miran
283f716593
update required mdbook versions (#5125)
I couldn't install those packages with the versions previously stated.
The new versions install successfully,
and `make publish-book` works without errors.
2023-06-26 16:04:01 +00:00
tersec
a2bdc2d723
revert 2023-06-26 08:38:05 +00:00
tersec
dbd7611983
not a real change 2023-06-26 08:36:39 +00:00
tersec
ba597ef0a2
per-validator payload builder configuration (#5062) 2023-06-25 12:00:17 +00:00