Commit Graph

5356 Commits

Author SHA1 Message Date
tersec 80773d7278
add gsf5 to devnets (#3899) 2022-07-23 16:51:08 +02:00
Etan Kissling fd4cf35c20
fix concurrent Jenkins stages (#3904)
The ports for the concurrently executing REST and Minimal testnet clash,
leading to some CI failures since #3827 introduced further concurrency.
Adjusting the ports to be distinct across various tests should fix this.
2022-07-23 14:28:10 +00:00
Etan Kissling 3bc42994e4
update to latest LC test format (#3879)
The EF test format for the LC sync protocol is modified to verify checks
after each step: https://github.com/ethereum/consensus-specs/pull/2938 -
The test runner is updated accordingly.
2022-07-23 05:54:01 +00:00
Etan Kissling c3f9844de1
do not disconnect on receiving canceled request (#3869)
LC cancels concurrent requests if one peer sent a correct response and
waiting for other peers is no longer useful. On the server side this
resulted in a descore (-500) and a likely disconnect. The behaviour is
changed to match `UnexpectedEOF`, `PotentiallyExpectedEOF` handling that
return an error response without disconnecting from the peer.
2022-07-23 04:20:49 +00:00
Jacek Sieka dbd3d02e63
Migrate docs to mkdocs (#3900)
`mkdocs` works with markdown similar to `mdbook` but is generally more
pleasing to the eye and has several nice UX features.

This PR does the bulk of the transition - likely, a followup would be
needed to fully make use of the extra features and navigation.

Book pages have been kept url-compatible, meaning that for the most
part, old links should continue to work!

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-22 21:47:24 +02:00
Jacek Sieka ddb7a3ecf0
polish intro 2022-07-22 18:18:31 +02:00
tersec 615f366dcc
update to supported macOS GitHub CI image (#3898) 2022-07-22 07:08:21 +00:00
Jakub Sokołowski c33989e490
ci: refactor Jenkinsfile to be a pipeline (#3827)
Changes:
- Name local testnet output folders same as the `make` taget.
- Move both `Jenkinsfile`s to `ci` folder to avoid cluttering repo root.
- Separate builds by platform so logs from macos and linux hosts don't get mixed.
- Detect platform and architecture from Jenkins Job path to use one Jenkinsfile.
- Divide shell commands into as many stages as possible to make debugging easier.
- Generalize running testnets via a `launchLocalTestnet()` Groovy method.
- Handle uploading of results of running testnets stage-by-stage basis.
- Use `catchError()` to upload test results while marking job as failed.
- Abort previously started PR build jobs usin `disableConcurrentBuilds()`.
- Throttle jobs using the new `throttleJobProperty()` function.

Builds:
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/linux/job/x86_64/
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/macos/job/x86_64/
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/macos/job/aarch64/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-22 01:34:31 +03:00
Zahary Karadjov 1a1553fee8
Bump eth2-networks to include the Prater TTD 2022-07-21 22:30:17 +03:00
Etan Kissling 322d47078b
classify prater as merged network (#3887)
Moves `prater` to the group of to be merged testnets.
2022-07-21 21:40:25 +03:00
tersec 2f77f05a1a
optimistic block gossip validation (#3876) 2022-07-21 21:39:43 +03:00
tersec f4208cfb23
opportunistically even less async optimistic sync (#3880) 2022-07-21 21:26:36 +03:00
Jacek Sieka f98e9ec8bc
update docs (#3890)
* update docs

* introduce mdbook-admonish for nice looking callouts
* new section on data directory
* recommend source build for advanced users and direct the rest to
binaries
* more strongly highlight that execution client is needed
* write an actual deposit guide
* remove cruft / fix links / etc
2022-07-21 21:19:47 +03:00
Miran 76fa4772fb
fix a typo in a link (#3897)
Before the fix, the link would take you to the non-existing
https://nimbus.guide/prater.beaconcha.in
2022-07-21 21:08:37 +03:00
Eugene Kabanov c3d3397843
VC: doppelganger protection (#3877)
* Improve fallback_service.

* Improve logging in fallback_service.

* Apply signal handling for all stages.

* Fix some logging statements.

* Add doppelganger REST api endpoint.
Add some structures to VC.

* Add client API call implementation.

* Initial fix & refactor onceToAll()
Add doppelganger service.
Add doppelganger helpers.

* Add doppelganger checks.

* Move doppelganger log messages to higher levels.

* Fix firstSuccess().

* Bump chronos.

* Post rebase fixes.

* Proper chronos bump.

* Address review comments.

* Attempt to fix finalization test issue.

* Fix nimbus_signing_node.

* Mark validators which are added at GENESIS_SLOT in GENESIS_EPOCH as passed doppelganger validation.

* Do not send empty requests to server.

* Fix log statement.

* Address review comments and re-raise cancellations.

Co-authored-by: zah <zahary@gmail.com>
2022-07-21 19:54:07 +03:00
Zahary Karadjov 751ddd62f8
Merge branch 'stable' into unstable 2022-07-21 14:20:33 +03:00
Jacek Sieka 3d7a35c116
`--era-dir` option (#3895)
* `--era-dir` option

hidden, while the feature is under development

* Update beacon_chain/conf.nim

Co-authored-by: zah <zahary@gmail.com>
2022-07-21 14:07:19 +03:00
Etan Kissling 5dcfb0c4e7
avoid packing attestations from other forks (#3893)
When there is heavy forking, proposals may get missed due to including
attestations from different forks that later fail verification.
Checking attestation signatures when building blocks should fix this.
2022-07-21 14:04:56 +03:00
Etan Kissling 735c1df62f
add strict mode to light client processor (#3894)
The light client sync protocol employs heuristics to ensure it does not
become stuck during non-finality or low sync committee participation.
These can enable use cases that prefer availability of recent data
over security. For our syncing use case, though, security is preferred.
An option is added to light client processor to configure this tradeoff.
2022-07-21 11:16:10 +02:00
Jacek Sieka 24d8401054
bumps (#3892)
various ABI fixes
2022-07-20 20:20:34 +02:00
Etan Kissling d6adf09e65
cleanup magic strings (#3891)
Replaces hardcoded IP address strings with constants.
Adjusts some LC log levels.
Removes no longer used `RpcServer` type.
2022-07-20 18:17:21 +00:00
tersec df660292a4
bump nim-metrics for styleCheck compatibility (#3889) 2022-07-20 10:13:32 +02:00
Miran 8ebd5a5fe8
fix some warnings (#3886) 2022-07-18 19:17:11 +00:00
tersec 3dc04bc48f
bump nimcrypto for styleCheck compatibility (#3885) 2022-07-18 17:13:25 +00:00
Jacek Sieka 7dff72c6bf
bumps (#3882)
* fix UB in leb128 shift
* bearssl spellings
2022-07-18 07:17:53 +02:00
zah e6988c1cc9
Bump nim-json-serialization (#3878) 2022-07-15 14:36:03 +00:00
zah 20d45e69b5
Re-enabled requireAllFields after a fix in nim-json-serialization (#3871)
* Re-enabled requireAllFields after a fix in nim-json-serialization

The problem was that `Option[T]` fields were not treated as optional
when requireAllFields is set to true. This is now fixed in NJS.

* Add makefile targets for recreating the Jenkins simulation runs

* Fix a discrepancy with the REST spec
2022-07-15 03:19:19 +03:00
Zahary Karadjov 798fa69647
[skip ci] remove an unused variable 2022-07-15 00:34:04 +03:00
kayla-henrie 6ff00a5694
[README] Add GitPOAP Badge to Display Number of Minted GitPOAPs for Contributors (#3868) 2022-07-15 00:14:21 +03:00
Eugene Kabanov d4bafdf5a4
VC: cancellation hot-fixes. (#3875)
* Fix cancellation issues.
* Add exitEvent which will allow gracefully shutdown validator client.
* Fix firstSuccessTimeout() template.
* Fix service names.
* Modify waitOnlineNodes to include timeout parameter.
2022-07-15 00:11:25 +03:00
zah a517e8718c
Allow the user to use 'goerli' instead of 'prater' (#3874) 2022-07-14 20:07:16 +00:00
zah 2bd5d03743
Fix a VC crash observed in the local_testnet simulation (#3872)
It's not quite clear why this condition was triggered in the local
simulation, but it seems a viable scenario after the Keymanager API
is integrated in the validator client.

The user can temporarily remove all validator keys from a running
client before adding another set of keys.
2022-07-14 21:48:04 +03:00
Etan Kissling a6deacd878
allow driving EL with LC (#3865)
Adds the `--web3-url` launch argument to `nimbus_light_client` to enable
driving the EL with the optimistic head obtained from LC sync protocol.
This will keep issuing `newPayload` / `forkChoiceUpdated` requests for
new blocks, marking them as optimistic. `ZERO_HASH` is reported as the
finalized block for now.
2022-07-14 04:07:40 +00:00
Etan Kissling 47a1b11d5d
revert `requireAllFields` regression (#3870)
#3864 introduced a regression by turning on `requireAllFields` globally
for JSON parsing. Certain endpoints such as `RestSyncInfo` have optional
fields that do not parse correctly without additional changes. This is
reverted for now to restore previous behaviour and unblock CI testing.
2022-07-14 03:33:15 +00:00
Etan Kissling ee4e4ddeab
cleanup `getPersistentNetKeys` for better reuse (#3859)
Allow using `getPersistentNetKeys` without passing a `config`, and reuse
local helpers in its implementation.
2022-07-13 23:26:16 +02:00
zah 806536a040
[Keymanager API] Support for the feerecipient end-points (#3864)
Other changes:

* The Keymanager error responses differ from the Beacon API responses.
  'keymanagerApiError' replaces the former usages of 'jsonError'.

* Return status code 401 and 403 for authorization errors in accordance
  to the spec.

* Eliminate inconsistencies in the REST JSON parsing. Some of the code
  paths allowed missing fields.

* Added logging of serialization failure details at DEBUG level.
2022-07-13 17:45:04 +03:00
tersec 456b5ebc48
add msf9 to devnets (#3867) 2022-07-13 17:44:40 +03:00
Eugene Kabanov 263a2ffa14
Validator client various fixes. (#3840)
* Improve fallback_service.
* Fix nextAction negative time issue.
* Improve logging in fallback_service.
* Improve logging in sync_committee_service.
* Prepare all services for cancellation.
* Signals handlers for validator client
* Address #3800

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2022-07-13 17:43:57 +03:00
tersec 06c8e10ae2
move consensus_manager to consensus_object_pools (#3852) 2022-07-13 14:13:54 +00:00
tersec ce6cbd84e2
rename verifyFinalization internal flag to strictVerification (#3866)
* rename verifyFinalization internal flag to strictVerification

* Update beacon_chain/extras.nim

Co-authored-by: Etan Kissling <etan@status.im>

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-13 13:48:09 +00:00
Etan Kissling 22e77131f8
export `DepositContractSnapshot` in eth1 monitor (#3860)
Avoid requiring explicit `beacon_chain_db` import in eth1 monitor use.
2022-07-13 01:45:48 +00:00
tersec b164f6357b
handle discrepancy between subnet tracking/nextaction tracking (#3862)
* handle discrepancy between subnet tracking/nextaction tracking

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: Etan Kissling <etan@status.im>

* consistency

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-13 02:03:07 +03:00
tersec e0e7af7aff
MEV boost script for ropsten and sepolia builder network (#3851)
* MEV boost script for ropsten and sepolia builder network

* networkInfo not a global variable
2022-07-12 20:50:12 +03:00
tersec 8a9e6ab8a4
enforce epoch alignment for --finalized-checkpoint-state/block (#3858)
* enforce epoch alignment for --finalized-checkpoint-state/block

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: Etan Kissling <etan@status.im>

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-12 15:09:56 +00:00
Etan Kissling 20491560b6
re-enable LC in local testnet (#3857)
`a48d741022f6b0da1bb679e0ede4e38c019242cf` disabled LC in local testnet
as an undocumented side effect. Re-enabling for more thorough testing,
and added handling of LC with `--eth2-docker-image`.
2022-07-12 11:38:50 +00:00
Etan Kissling 99eca42b13
introduce `loadJwtSecret` helper (#3856)
Extract repeated code to load the JWT secret file into a separate func.
2022-07-12 10:08:52 +00:00
tersec 95b2596dfe
track the SyncCommittee participation in slot end logs (#3854)
* track the SyncCommittee period in slot end logs

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: Etan Kissling <etan@status.im>

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-12 09:00:39 +00:00
tersec ee27274078
update Kiln book chapter (#3853)
* update Kiln book chapter

* remove reference to kiln branch of nethermind
2022-07-12 07:51:40 +00:00
Etan Kissling 571577c662
mark Sepolia as merge completed (#3848)
Now that Sepolia successfully completed the merge, move it to the
corresponding group to enable in-development LC feature testing.
2022-07-10 17:49:12 -07:00
Etan Kissling f9536fd14b
fork choice cleanups (#3849)
Removes a few extra-ambitious templates to make `self` updates explicit,
and moves the `FinalityCheckpoints` type from `base` to `helpers` as it
is an additional Nimbus specific type not defined by spec.
2022-07-10 15:26:29 +00:00