Commit Graph

4599 Commits

Author SHA1 Message Date
Etan Kissling 98a533c781
resume EL sync at correct block on reconnect (#3957)
When connection to the EL is lost as part of EL deposits importing, the
targeted block range to sync would reset. This is changed to properly
remember import progress across reconnects.
2022-08-12 16:48:33 +03:00
Nikolay Mitev d07d7b1c48 Include auxiliary tools to nightly builds
Create stable link (withouth hash) so that scripts can always get the latest nightly build from the same url
2022-08-11 17:57:44 +03:00
zah d64c17ffc3
Minor post-merge cleanups (#3945)
https://github.com/status-im/nimbus-eth2/pull/3944

The use of nested `awaitWithRetries` calls would have
resulted in an unexpected number of retries (3x3).
We now use regular `await` in outer layer to avoid the problem.

https://github.com/status-im/nimbus-eth2/pull/3943

The new code has an invariant that the `headMerkleizer` field in
the `Eth1Chain` is always kept in sync with the blocks stored in
the chain.

This invariant is now enforced better by doing the necessary merkleizer updates
in the `Eth1Chain.addBlock` function, in the `Eth1Chain.init` function and in the
`Eth1Chain.reset` function.
2022-08-10 12:31:10 +00:00
Etan Kissling ede83b1805
consider blocks with deposits in eth1 data vote (#3944)
When importing blocks with deposits from the EL, the timestamp is never
initialized for them. Therefore, only blocks without deposits (for which
the timestamp is obtained) are considered for `is_candidate_block`.
This is fixed by also importing timestamps for blocks with deposits.
2022-08-10 01:14:02 +03:00
Etan Kissling 4ef621f926
fix obtaining deposits after connection loss (#3943)
* fix obtaining deposits after connection loss

When an error occurs during Eth1 deposits import, the already imported
blocks are kept while the connection to the EL is re-established.
However, the corresponding merkleizer is not persisted, leading to any
future deposits no longer being properly imported. This is quite common
when syncing a fresh Nimbus instance against an already-synced Geth EL.
Fixed by persisting the head merkleizer together with the blocks.
2022-08-10 00:32:34 +03:00
zah 06a5c67e62
The body of prepare_beacon_proposer is an array (Fixes #3920) (#3938) 2022-08-09 12:53:02 +03:00
zah dc50abbc90
Implement a missing ingnore rule for sync committee contributions (#3941) 2022-08-09 12:52:11 +03:00
Eugene Kabanov 250f7b4bdf
Exclusive keystore locking (#3907) 2022-08-08 00:53:20 +03:00
Eugene Kabanov fe5435ed96
Address #3624 (#3934) 2022-08-06 11:55:40 +00:00
zah 8273b3d909
Keep CLI options consistent by removing the '-enable' suffix from the outliers (#3928) 2022-08-05 17:38:26 +02:00
tersec 52b32c13e1
update macOS baseline x86 CPU architecture to Ivy Bridge (#3929) 2022-08-02 19:02:56 +00:00
Etan Kissling 9c6a4316aa
document LC data serving options (#3922)
Adds a documentation page for configuring LC data serving.
2022-08-02 12:23:03 +00:00
tersec 2240594ed8
beacon_chain_db: proc -> func (#3931) 2022-08-01 16:17:06 +00:00
tersec 73900172cb
update Goerli/Prater documentation and remove Kiln documentation (#3925) 2022-08-01 09:52:36 +03:00
tersec d62d13a23c
MEV block proposal (#3883)
* MEV validator registration

* add nearby canary to detect new beacon chain forks

* remove special MEV graffiti

* web3signer support

* fix trace logging

* Nim 1.2 needs raises Defect

* use template rather than proc in REST JSON parsing

* use --payload-builder-enable and --payload-builder-url

* explicitly default MEV to disabled

* explicitly empty default value for payload builder URL

* revert attestation pool to unstable version
2022-08-01 09:41:47 +03:00
Taneli Hukkinen 17bf42316e
Fix typo in log: canoncial->canonical (#3927) 2022-08-01 00:32:08 +00:00
dependabot[bot] 3c96b9c69b
Bump mistune from 0.8.4 to 2.0.3 in /ncli (#3924)
Bumps [mistune](https://github.com/lepture/mistune) from 0.8.4 to 2.0.3.
- [Release notes](https://github.com/lepture/mistune/releases)
- [Changelog](https://github.com/lepture/mistune/blob/master/docs/changes.rst)
- [Commits](https://github.com/lepture/mistune/compare/v0.8.4...v2.0.3)

---
updated-dependencies:
- dependency-name: mistune
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-30 07:01:54 +00:00
tersec a8cefee4c5
remove obsolete merge documentation (#3923) 2022-07-29 18:02:09 +00:00
Etan Kissling 449848451d
use LC features on all networks (#3921)
Prepares removing functionality to restrict LC features to testnets.
Rest of functions is removed later to have a quick way of reverting.
2022-07-29 15:37:47 +00:00
Miran dfd4afc9f2
compatibility with Nim 1.4+ (#3888) 2022-07-29 10:53:42 +00:00
Etan Kissling 3ec7982293
update light client protocol version (#3550)
* Use final `v1` version for light client protocols
* Unhide LC data collection options
* Default enable LC data serving
* rm unneeded import
* Connect to EL on startup
* Add docs for LC based EL sync
2022-07-29 11:45:39 +03:00
Eugene Kabanov ce9e50e275
VC: metrics (#3915)
* Initial commit.
Enable MetricsHttpServerRef and configuration.

* Add metrics.

* Add headers.
Add compilation issue fixes.
2022-07-29 11:36:20 +03:00
Zahary Karadjov 9b081e524c
Merge branch 'stable' into unstable 2022-07-29 11:28:43 +03:00
Zahary Karadjov 5951df4193
Switch back to Nim 1.2
We've discovered a critical `lent` issue affecting Nim 1.6:
https://github.com/nim-lang/Nim/issues/20107
2022-07-29 11:26:36 +03:00
Zahary Karadjov 75ccfcb26f
Ship the Validator Client (Currently in BETA) 2022-07-27 20:16:02 +03:00
Zahary Karadjov 64e791be66
Revert "avoid packing attestations from other forks (#3893)"
This reverts commit 5dcfb0c4e7.
2022-07-27 20:14:40 +03:00
tersec b7f907a64a
bump nim-zxcvbn for styleCheck compatibility (#3919) 2022-07-27 14:25:09 +03:00
tersec 9d41fbfded
bump nim-presto for styleCheck compatibility (#3918) 2022-07-27 14:24:45 +03:00
Zahary Karadjov 4637669b6b
Merge branch 'stable' into unstable 2022-07-26 23:54:46 +03:00
Zahary Karadjov 7cac6f46db
Version 22.7.0 2022-07-26 23:53:16 +03:00
Zahary Karadjov be68f91f5c
Honor the WEB3_URL env var in the run-node scripts to preserve backward compatibility 2022-07-26 14:04:07 +03:00
Jakub Sokołowski d2d6e632a8
fix tesnet port conflicts on Jenkins CI
A fix for a bug triggered by recent `Jenkinsfile` refactoring done in:
https://github.com/status-im/nimbus-eth2/pull/3827

Which due to a big in Jenkins Throttling plugin caused jobs to start
running in parallel on the same host despite global configuration that
is supposed to block this:
https://issues.jenkins.io/browse/JENKINS-49173
https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/68

An attempt to fix this was made in this PR:
https://github.com/status-im/nimbus-eth2/pull/3913

But it was ineffective due to bugs in the Throttle plugin.

As a result semi-random testnet launches would fail with errors like this:
```
./scripts/launch_local_testnet.sh: line 1026: 58977 Killed: 9   ${BEACON_NODE_COMMAND} ...
```
The culprit was the old process cleanup in `scripts/launch_local_testnet.sh`:
```
+ make local-testnet-mainnet
Found old process listening on port 7001, with PID 58977. Killing it.
Found old process listening on port 7002, with PID 59024. Killing it.
Found old process listening on port 7003, withu PID 59027. Killing it.
Found old process listening on port 7004, with PID 59030. Killing it.
```
Which was triggered due to use of immediate assignment for `EXECUTOR_NUMBER`:
```
EXECUTOR_NUMBER := 0
```
Which cause the `EXECUTOR_NUMBER` value set by Jenkins to be ignored.

For more details see:
https://www.gnu.org/software/make/manual/html_node/Flavors.html#Flavors

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-26 11:54:02 +02:00
Jakub Sokołowski 3f30eb51dd
launch_local_testnet: show processes killed by pkill
I want to verify that processes being killed are not from other parallel jobs.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-26 11:54:02 +02:00
tersec b7ed0aa235
bump nim-websock for styleCheck compatibility (#3909) 2022-07-26 12:20:36 +03:00
tersec ee24beee30
bump nim-stint for styleCheck compatibility (#3908) 2022-07-26 12:19:55 +03:00
zah c5d89bb46e
Switch to Nim 1.6.7 (#3902) 2022-07-26 12:19:32 +03:00
tersec 6b2abd01fb
bump nim-eth for styleCheck compatibility (#3896) 2022-07-25 23:15:46 +03:00
zah cd04f27c37
Implement the /eth/v1/validator/prepare_beacon_proposer end-point (#3901) 2022-07-25 23:12:53 +03:00
zah 7d27e10315
Avoid potential stalls in the Eth1 monitor code due to missing timeouts (#3905)
* Avoid potential stalls in the Eth1 monitor code due to missing timeouts

* 1 second timeout for exchange-configuration
2022-07-25 22:23:02 +03:00
Taneli Hukkinen 10309e3d27
Move from --rpc to --rest in docker-compose examples (#3910)
* Move from --rpc to --rest in docker-compose examples

* Change REST port to 5052
2022-07-25 21:45:02 +03:00
Jakub Sokołowski 5b66cd141e
ci: specify explicit throttle limits for Jenkins (#3913)
Otherwise they appear to not have any effect:
https://issues.jenkins.io/browse/JENKINS-49173
https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/68

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-25 16:24:02 +02:00
Jacek Sieka dcea09ea5c
book edits (#3903)
* Reorganise menu, putting tabs on top
* Use tabs for specific content
* cleanups and consistency edits
2022-07-24 22:13:47 +02:00
Etan Kissling e4051710be
add msf10 to devnets (#3906) 2022-07-23 18:10:07 +00:00
tersec 80773d7278
add gsf5 to devnets (#3899) 2022-07-23 16:51:08 +02:00
Etan Kissling fd4cf35c20
fix concurrent Jenkins stages (#3904)
The ports for the concurrently executing REST and Minimal testnet clash,
leading to some CI failures since #3827 introduced further concurrency.
Adjusting the ports to be distinct across various tests should fix this.
2022-07-23 14:28:10 +00:00
Etan Kissling 3bc42994e4
update to latest LC test format (#3879)
The EF test format for the LC sync protocol is modified to verify checks
after each step: https://github.com/ethereum/consensus-specs/pull/2938 -
The test runner is updated accordingly.
2022-07-23 05:54:01 +00:00
Etan Kissling c3f9844de1
do not disconnect on receiving canceled request (#3869)
LC cancels concurrent requests if one peer sent a correct response and
waiting for other peers is no longer useful. On the server side this
resulted in a descore (-500) and a likely disconnect. The behaviour is
changed to match `UnexpectedEOF`, `PotentiallyExpectedEOF` handling that
return an error response without disconnecting from the peer.
2022-07-23 04:20:49 +00:00
Jacek Sieka dbd3d02e63
Migrate docs to mkdocs (#3900)
`mkdocs` works with markdown similar to `mdbook` but is generally more
pleasing to the eye and has several nice UX features.

This PR does the bulk of the transition - likely, a followup would be
needed to fully make use of the extra features and navigation.

Book pages have been kept url-compatible, meaning that for the most
part, old links should continue to work!

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-22 21:47:24 +02:00
Jacek Sieka ddb7a3ecf0
polish intro 2022-07-22 18:18:31 +02:00
tersec 615f366dcc
update to supported macOS GitHub CI image (#3898) 2022-07-22 07:08:21 +00:00