Commit Graph

352 Commits

Author SHA1 Message Date
tersec 5677816df3
remove obsolete merge test vector test bash script (#4135)
* remove obsolete merge test vector test bash script

* remove reference to removed merge test vector script
2022-09-16 22:49:39 +02:00
Mamy Ratsimbazafy f21495b9d4
Use engine API port in systemd.service fix #4098 (#4099) 2022-09-09 12:18:02 +00:00
Etan Kissling 317e91bfc1
support jwtsecret in `launch_local_testnet` (#4097)
To allow testing more thoroughly with connected EL, pass `jwtsecret`
as part of running local testnets.
2022-09-08 15:22:31 +02:00
Etan Kissling 613f4a9a50
accelerate EL sync with LC with `--sync-light-client` (#4041)
When the BN-embedded LC makes sync progress, pass the corresponding
execution block hash to the EL via `engine_forkchoiceUpdatedV1`.
This allows the EL to sync to wall slot while the chain DAG is behind.
Renamed `--light-client` to `--sync-light-client` for clarity, and
`--light-client-trusted-block-root` to `--trusted-block-root` for
consistency with `nimbus_light_client`.

Note that this does not work well in practice at this time:
- Geth sticks to the optimistic sync:
  "Ignoring payload while snap syncing" (when passing the LC head)
  "Forkchoice requested unknown head" (when updating to LC head)
- Nethermind syncs to LC head but does not report ancestors as VALID,
  so the main forward sync is still stuck in optimistic mode:
  "Pre-pivot block, ignored and returned Syncing"

To aid EL client teams in fixing those issues, having this available
as a hidden option is still useful.
2022-08-29 12:16:35 +00:00
tersec 61dc296046
update engine API spec ref URLs from alpha.9 to beta.1 (#4030)
* update engine API spec ref URLs from alpha.9 to beta.1

* require exactly 256-bit JWT keys
2022-08-26 13:44:50 +03:00
tersec 184c3de02d
remove MEV test script (#4031) 2022-08-26 10:02:07 +02:00
zah 4e41ed1d5a
Require properly configured Engine API connection after the merge (#4006) 2022-08-22 22:44:40 +03:00
zah b1ac9c9fe4
Fix a potential segfault and various potential stalls (#4003)
* Fixes a segfault during block production when the Keymanager API
  is disabled. The Keymanager is now disabled on half of the local
  testnet nodes to catch such problems in the future.

* Fixes multiple potential stalls from REST requests being done
  without a timeout. From practice, we know that such requests
  can hang forever if not cancelled with a timeout. At best,
  this would be a resource leak, at worst, it may lead to a
  full stall of the client and missed validator duties.

* Changes some Options usages to Opt (for easier use of valueOr)
2022-08-19 21:51:30 +00:00
zah fca20e08d6
Keymanager API for the validator client (#3976)
* Keymanager API for the validator client
* Properly treat the 'description' field as optional when loading Keystores
* Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response ()

Fixes #3940
Fixes #3964
Closes #3884 by adding test
2022-08-19 13:30:07 +03:00
Jacek Sieka 255be39e69
packaging updates (#3974)
* packaging updates

* one package per binary (nimbus_beacon_node, nimbus_validator_client)
* use `-` in package name (`_` is separating the version)
* don't include (un)installation scripts in package
* default metrics port 8108 for vc
* fix several upgrade/install errors in scripts
* add JWT option to service files
* don't attempt to remove user on purge
2022-08-17 13:26:31 +03:00
zah ca3245c4f0
Doppelganger exit code changed from 1031 to 129 (addresses #3973) (#3977) 2022-08-17 08:13:55 +02:00
Nikolay Mitev 5c91d29df0 Trivial: add wait_for_port function to script 2022-08-16 12:39:27 +03:00
Nikolay Mitev 33546f0fa9 Trivial: Fix typo 2022-08-15 16:46:14 +03:00
Nikolay Mitev 9e6d9b955d Trivial: Make NIMBUS_EL_BINARY customizable 2022-08-14 15:51:42 +03:00
Nikolay Mitev 607de676cb
launch_local_testnet script: add options to download eth2 binary (#3958)
Some refactoring and cleanup
2022-08-12 17:41:11 +03:00
Nikolay Mitev d07d7b1c48 Include auxiliary tools to nightly builds
Create stable link (withouth hash) so that scripts can always get the latest nightly build from the same url
2022-08-11 17:57:44 +03:00
zah 8273b3d909
Keep CLI options consistent by removing the '-enable' suffix from the outliers (#3928) 2022-08-05 17:38:26 +02:00
Etan Kissling 9c6a4316aa
document LC data serving options (#3922)
Adds a documentation page for configuring LC data serving.
2022-08-02 12:23:03 +00:00
Miran dfd4afc9f2
compatibility with Nim 1.4+ (#3888) 2022-07-29 10:53:42 +00:00
Etan Kissling 3ec7982293
update light client protocol version (#3550)
* Use final `v1` version for light client protocols
* Unhide LC data collection options
* Default enable LC data serving
* rm unneeded import
* Connect to EL on startup
* Add docs for LC based EL sync
2022-07-29 11:45:39 +03:00
Zahary Karadjov 4637669b6b
Merge branch 'stable' into unstable 2022-07-26 23:54:46 +03:00
Zahary Karadjov be68f91f5c
Honor the WEB3_URL env var in the run-node scripts to preserve backward compatibility 2022-07-26 14:04:07 +03:00
Jakub Sokołowski 3f30eb51dd
launch_local_testnet: show processes killed by pkill
I want to verify that processes being killed are not from other parallel jobs.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-26 11:54:02 +02:00
Etan Kissling fd4cf35c20
fix concurrent Jenkins stages (#3904)
The ports for the concurrently executing REST and Minimal testnet clash,
leading to some CI failures since #3827 introduced further concurrency.
Adjusting the ports to be distinct across various tests should fix this.
2022-07-23 14:28:10 +00:00
Etan Kissling a6deacd878
allow driving EL with LC (#3865)
Adds the `--web3-url` launch argument to `nimbus_light_client` to enable
driving the EL with the optimistic head obtained from LC sync protocol.
This will keep issuing `newPayload` / `forkChoiceUpdated` requests for
new blocks, marking them as optimistic. `ZERO_HASH` is reported as the
finalized block for now.
2022-07-14 04:07:40 +00:00
Etan Kissling 22e77131f8
export `DepositContractSnapshot` in eth1 monitor (#3860)
Avoid requiring explicit `beacon_chain_db` import in eth1 monitor use.
2022-07-13 01:45:48 +00:00
tersec e0e7af7aff
MEV boost script for ropsten and sepolia builder network (#3851)
* MEV boost script for ropsten and sepolia builder network

* networkInfo not a global variable
2022-07-12 20:50:12 +03:00
Etan Kissling 20491560b6
re-enable LC in local testnet (#3857)
`a48d741022f6b0da1bb679e0ede4e38c019242cf` disabled LC in local testnet
as an undocumented side effect. Re-enabling for more thorough testing,
and added handling of LC with `--eth2-docker-image`.
2022-07-12 11:38:50 +00:00
tersec 19bf7f1599
check in MEV boost test script that payload gets unblinded (#3838) 2022-07-05 04:13:07 +00:00
tersec 31772a2ac2
don't use JWT if one hasn't set up a JWT secret (#3822) 2022-06-29 07:36:22 +00:00
Jacek Sieka ff12c7f9ce
remove web3 url prompt in launcher script (#3810)
* remove web3 url prompt in launcher script

The interactive prompt for web3 has outlived its utility as we now load
url:s from command line params and config files, preventing the prompt
from correctly detecting when it's needed.

Also, after the merge, a JWT secret will (likely) be needed.

* log notice when web3 url is missing
* fix docs to not mention default that doesn't exist
* fix scripts to properly quote arguments
2022-06-28 23:49:52 +03:00
Jacek Sieka 0152f4ade5
Make deb/rpm packages when creating release (#3796)
* Make deb/rpm packages when creating release

A step towards #3782

* fix arch for arm
2022-06-28 23:34:18 +03:00
Zahary Karadjov 73227508db
Fix regressions in launch_local_simulation 2022-06-28 19:22:06 +03:00
Nikolay Mitev a48d741022 Update local-testnet-simulation with changes from kiln-dev-auth branch 2022-06-28 13:33:49 +02:00
zah b318a74d83
Add support for launching Geth and Nimbus-eth1 nodes in launch_local_testnet (#3790)
* Add support for launching Geth and Nimbus-eth1 nodes in launch_local_simulation

* Prevent undefined variables errors when Nimbus and Geth are not launched

Co-authored-by: Nikolay Mitev <github@hmel.org>
2022-06-27 11:34:12 +03:00
tersec 7e61198260
fix merge test scripts (#3781) 2022-06-20 18:03:57 +00:00
Etan Kissling 9dcbc44fc8
fix `--light-clients` option in `launch_local_testnet` (#3779)
`launch_local_testnet` was not recognizing `--light-clients` correctly.
2022-06-19 20:56:34 +03:00
tersec 8eb5d5de09
use ZERO_HASH for default(Eth2Digest)/Eth2Digest() in func calls (#3770) 2022-06-18 04:57:37 +00:00
Jacek Sieka c48ea3cdc8
book updates (#3763)
* systemd: use same service file as package
* systemd: recommend setting up a user
* clean up obsolete / ephemeral notes
* eth1 -> execution, in many places
* clean up docker instructions to match the experience of running the
runner script
* add basic instructions / links for execution clients other than geth
* clear a few more references to JSON-RPC, update to REST
* update hardware requirements with execution client extras
* start talking about merge requirements (this needs a dedicated page)
* note that binaries do exist, where relevant
* list gitcoin under donations
* update syncing docs
* `NOT` is for `NOTICE` in logs
* update rpi hardware recommendations
* prevent automatic doppelganger restarts in systemd service file
2022-06-17 18:28:13 +03:00
Etan Kissling 52ba4f7999
rename light client config parameters (#3740)
For consistency with other options, use a common prefix for light client
data configuration options.

* `--serve-light-client-data` --> `--light-client-data-serve`
* `--import-light-client-data` --> `--light-client-data-import-mode`

No deprecation of the old identifiers as they were only sparingly used
and all usage can be easily updated without interferance.
2022-06-14 12:03:39 +03:00
Etan Kissling 21a1f7eeb7
error handling for local testnet REST query (#3739)
When querying `ALTAIR_FORK_EPOCH` fails, the `launch_local_testnet`
script got stuck because comparing `CURRENT_FORK_EPOCH` against it
would keep failing with `[: : integer expression expected`.
Querying `ALTAIR_FORK_EPOCH` is now retried until a number is returned.
2022-06-14 08:38:39 +00:00
tersec 0cedb1b6a7
seq(1) can count backwards on macOS; catch undefined variable use (#3735) 2022-06-13 20:01:39 +03:00
Etan Kissling 15967c4076
keep track of latest blocks for optimistic sync (#3715)
When launched with `--light-client-enable` the latest blocks are fetched
and optimistic candidate blocks are passed to a callback (log for now).
This helps accelerate syncing in the future (optimistic sync).
2022-06-10 14:16:37 +00:00
tersec 1262e50fa5
document MEV/mergemock test script (#3694) 2022-06-06 16:55:02 +03:00
Etan Kissling 01efa93cf6
add light client (standalone) (#3653)
Introduces a new library for syncing using libp2p based light client
sync protocol, and adds a new `nimbus_light_client` executable that uses
this library for syncing. The new executable emits log messages when
new beacon block headers are received, and is integrated into testing.
2022-05-31 12:45:37 +02:00
tersec 92c1e75915
stop using --rpc in run-beacon-node and simulations (#3662) 2022-05-24 16:21:55 +00:00
Nikolay Mitev b3dc9e2c08
Update merge test vector script and documentation (#3638) 2022-05-16 16:12:32 +03:00
zah a2ba34f686
Implement all sync committee duties in the validator client (#3583)
Other changes:

* logtrace can now verify sync committee messages and contributions
* Many unnecessary use of pairs() have been removed for consistency
* Map 40x BN response codes to BeaconNodeStatus.Incompatible in the VC
2022-05-10 10:03:40 +00:00
zah 6d11ad6ce1
Support for distributed keystores with multiple remotes based on threshold signatures (#3616)
Other fixes:

* Fix bit rot in the `make prater-dev-deposit` target.
* Correct content-type in the responses of the Nimbus signing node
* Invalid JSON payload was being sent in the web3signer requests
2022-05-10 03:32:12 +03:00
Etan Kissling 89ac586bd4
serve light client data in CI / dev builds (#3487)
Adjust config for CI / dev builds to serve light client data by default:
`--serve-light-client-data=1 --import-light-client-data=only-new`
2022-03-12 22:12:18 +01:00