Commit Graph

266 Commits

Author SHA1 Message Date
Etan Kissling 0708fcd7cf
rm require engine API check (#4144)
The `eth1_monitor` check to require engine API from bellatrix onward
has issues in setups where the EL and CL are started simultaneously
because the EL may not be ready to answer requests by the time that the
check is performed. This can be observed, e.g., on Raspberry Pi 4 when
using Besu as the EL client. Now that the merge transition happened, the
check is also not that useful anymore, as users have other ways to know
that their setup is not working correctly (e.g., repeated exchange logs)
2022-09-19 23:47:46 +02:00
Etan Kissling abd6581b22
allow custom configs in Gnosis binary (#4134)
When running as Gnosis-chain binary the config was no longer adjustable.
Restores loading custom configs when running as Gnosis-chain binary,
as long as the following keys remain same:
- SLOTS_PER_EPOCH=16
- SECONDS_PER_SLOT=5
- BASE_REWARD_FACTOR=25
- EPOCHS_PER_SYNC_COMMITTEE_PERIOD=512

This allows running the Gnosis-chain binary on custom test networks.
2022-09-19 12:25:41 +03:00
tersec ad0d30093f
state/forkyState cleanup; spec URL updates; rm unused imports (#4052) 2022-08-31 13:29:34 +02:00
Etan Kissling 574b84f96f
add REST endpoint for fork choice context (#4042)
Implements a proposed REST endpoint for analyzing fork choice behaviour.
See https://github.com/ethereum/beacon-APIs/pull/232
2022-08-29 22:02:29 +00:00
Etan Kissling 613f4a9a50
accelerate EL sync with LC with `--sync-light-client` (#4041)
When the BN-embedded LC makes sync progress, pass the corresponding
execution block hash to the EL via `engine_forkchoiceUpdatedV1`.
This allows the EL to sync to wall slot while the chain DAG is behind.
Renamed `--light-client` to `--sync-light-client` for clarity, and
`--light-client-trusted-block-root` to `--trusted-block-root` for
consistency with `nimbus_light_client`.

Note that this does not work well in practice at this time:
- Geth sticks to the optimistic sync:
  "Ignoring payload while snap syncing" (when passing the LC head)
  "Forkchoice requested unknown head" (when updating to LC head)
- Nethermind syncs to LC head but does not report ancestors as VALID,
  so the main forward sync is still stuck in optimistic mode:
  "Pre-pivot block, ignored and returned Syncing"

To aid EL client teams in fixing those issues, having this available
as a hidden option is still useful.
2022-08-29 12:16:35 +00:00
tersec d7e9c334ac
document external block builder configuration (#4032)
* document external block builder configuration

* Update docs/the_nimbus_book/src/external-block-builder.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* unhide external payload builder options

* clarify builder API incentive misalignment

Co-authored-by: Jacek Sieka <jacek@status.im>
2022-08-29 12:59:12 +03:00
tersec 61dc296046
update engine API spec ref URLs from alpha.9 to beta.1 (#4030)
* update engine API spec ref URLs from alpha.9 to beta.1

* require exactly 256-bit JWT keys
2022-08-26 13:44:50 +03:00
zah 4e41ed1d5a
Require properly configured Engine API connection after the merge (#4006) 2022-08-22 22:44:40 +03:00
zah fca20e08d6
Keymanager API for the validator client (#3976)
* Keymanager API for the validator client
* Properly treat the 'description' field as optional when loading Keystores
* Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response ()

Fixes #3940
Fixes #3964
Closes #3884 by adding test
2022-08-19 13:30:07 +03:00
Jacek Sieka 255be39e69
packaging updates (#3974)
* packaging updates

* one package per binary (nimbus_beacon_node, nimbus_validator_client)
* use `-` in package name (`_` is separating the version)
* don't include (un)installation scripts in package
* default metrics port 8108 for vc
* fix several upgrade/install errors in scripts
* add JWT option to service files
* don't attempt to remove user on purge
2022-08-17 13:26:31 +03:00
zah 8273b3d909
Keep CLI options consistent by removing the '-enable' suffix from the outliers (#3928) 2022-08-05 17:38:26 +02:00
Etan Kissling 9c6a4316aa
document LC data serving options (#3922)
Adds a documentation page for configuring LC data serving.
2022-08-02 12:23:03 +00:00
tersec d62d13a23c
MEV block proposal (#3883)
* MEV validator registration

* add nearby canary to detect new beacon chain forks

* remove special MEV graffiti

* web3signer support

* fix trace logging

* Nim 1.2 needs raises Defect

* use template rather than proc in REST JSON parsing

* use --payload-builder-enable and --payload-builder-url

* explicitly default MEV to disabled

* explicitly empty default value for payload builder URL

* revert attestation pool to unstable version
2022-08-01 09:41:47 +03:00
Miran dfd4afc9f2
compatibility with Nim 1.4+ (#3888) 2022-07-29 10:53:42 +00:00
Etan Kissling 3ec7982293
update light client protocol version (#3550)
* Use final `v1` version for light client protocols
* Unhide LC data collection options
* Default enable LC data serving
* rm unneeded import
* Connect to EL on startup
* Add docs for LC based EL sync
2022-07-29 11:45:39 +03:00
Eugene Kabanov ce9e50e275
VC: metrics (#3915)
* Initial commit.
Enable MetricsHttpServerRef and configuration.

* Add metrics.

* Add headers.
Add compilation issue fixes.
2022-07-29 11:36:20 +03:00
Eugene Kabanov c3d3397843
VC: doppelganger protection (#3877)
* Improve fallback_service.

* Improve logging in fallback_service.

* Apply signal handling for all stages.

* Fix some logging statements.

* Add doppelganger REST api endpoint.
Add some structures to VC.

* Add client API call implementation.

* Initial fix & refactor onceToAll()
Add doppelganger service.
Add doppelganger helpers.

* Add doppelganger checks.

* Move doppelganger log messages to higher levels.

* Fix firstSuccess().

* Bump chronos.

* Post rebase fixes.

* Proper chronos bump.

* Address review comments.

* Attempt to fix finalization test issue.

* Fix nimbus_signing_node.

* Mark validators which are added at GENESIS_SLOT in GENESIS_EPOCH as passed doppelganger validation.

* Do not send empty requests to server.

* Fix log statement.

* Address review comments and re-raise cancellations.

Co-authored-by: zah <zahary@gmail.com>
2022-07-21 19:54:07 +03:00
Jacek Sieka 3d7a35c116
`--era-dir` option (#3895)
* `--era-dir` option

hidden, while the feature is under development

* Update beacon_chain/conf.nim

Co-authored-by: zah <zahary@gmail.com>
2022-07-21 14:07:19 +03:00
Etan Kissling d6adf09e65
cleanup magic strings (#3891)
Replaces hardcoded IP address strings with constants.
Adjusts some LC log levels.
Removes no longer used `RpcServer` type.
2022-07-20 18:17:21 +00:00
Etan Kissling a6deacd878
allow driving EL with LC (#3865)
Adds the `--web3-url` launch argument to `nimbus_light_client` to enable
driving the EL with the optimistic head obtained from LC sync protocol.
This will keep issuing `newPayload` / `forkChoiceUpdated` requests for
new blocks, marking them as optimistic. `ZERO_HASH` is reported as the
finalized block for now.
2022-07-14 04:07:40 +00:00
tersec ce6cbd84e2
rename verifyFinalization internal flag to strictVerification (#3866)
* rename verifyFinalization internal flag to strictVerification

* Update beacon_chain/extras.nim

Co-authored-by: Etan Kissling <etan@status.im>

Co-authored-by: Etan Kissling <etan@status.im>
2022-07-13 13:48:09 +00:00
Etan Kissling 99eca42b13
introduce `loadJwtSecret` helper (#3856)
Extract repeated code to load the JWT secret file into a separate func.
2022-07-12 10:08:52 +00:00
Etan Kissling aff53e962f
merge LC db into main BN db (#3832)
* merge LC db into main BN db

To treat derived LC data similar to derived state caches, merge it into
the main beacon node DB.

* shorten table names, group with lc prefix
2022-07-04 23:46:32 +03:00
Etan Kissling 499abd927f
persist LC data across restarts (#3823)
* persist LC data across restarts

With the Altair spec `LightClientUpdate` structure taking its final form
it is finally possible to persist LC data across restarts without having
to worry about data migration due to spec changes. A separate `lcdataV1`
database is created in the `caches` subdirectory to hold known LC data.
A full database with default settings (129 periods) uses <15 MB disk.

* extend LC data DB rationale

* wording

* add `isSupportedBySQLite` helper and explicit return

* remove redundant `return`
2022-06-30 13:04:39 +00:00
Jacek Sieka c145916414
cleanups (#3819)
* avoid circular panda imports
* move deposit merkleization helpers to spec/
* normalize validator signature helpers to spec names / params
* remove redundant functions for remote signing
2022-06-29 18:53:59 +02:00
Etan Kissling 91d543440a
add option to configure max historic LC data periods (#3799)
Adds a `--light-client-data-max-periods` option to override the number
of sync committee periods to retain light client data.
Raising it above the default enables archive nodes to serve full data.
Lowering below the default speeds up import times (still no persistence)
2022-06-27 13:24:38 +02:00
zah 69f505e2ba
Correct parsing of the `weak-subjectivity-checkpoint` parameter (#3765) 2022-06-17 14:55:03 +03:00
Etan Kissling 6144ecd95c
format pragmas consistently in `conf.nim` (#3750)
Aligns code style in `conf.nim`. Some pragmas lacked trailing `.`
2022-06-15 18:04:07 +00:00
Etan Kissling 52ba4f7999
rename light client config parameters (#3740)
For consistency with other options, use a common prefix for light client
data configuration options.

* `--serve-light-client-data` --> `--light-client-data-serve`
* `--import-light-client-data` --> `--light-client-data-import-mode`

No deprecation of the old identifiers as they were only sparingly used
and all usage can be easily updated without interferance.
2022-06-14 12:03:39 +03:00
Etan Kissling 15967c4076
keep track of latest blocks for optimistic sync (#3715)
When launched with `--light-client-enable` the latest blocks are fetched
and optimistic candidate blocks are passed to a callback (log for now).
This helps accelerate syncing in the future (optimistic sync).
2022-06-10 14:16:37 +00:00
Etan Kissling 72a46bd520
integrate light client into beacon node (#3557)
Adds a `LightClient` instance to the beacon node as preparation to
accelerate syncing in the future (optimistic sync).

- `--light-client-enable` turns on the feature
- `--light-client-trusted-block-root` configures block to start from

If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 19:01:11 +02:00
tersec ce143a1078
update CL spec URLs (#3690) 2022-06-01 15:52:45 +00:00
Jacek Sieka 6a0433cb08
REST parameter defaults (#3689)
* add default BN for VC
* unify "default" beacon node across commands
* clean up validator exit docs
2022-06-01 10:47:52 +00:00
tersec 48f9631ed1
remove JSON-RPC consensus layer API client (#3676)
* remove JSON-RPC consensus layer API client

* rm eth2_json_rpc_serialization
2022-05-31 11:05:15 +00:00
Etan Kissling 01efa93cf6
add light client (standalone) (#3653)
Introduces a new library for syncing using libp2p based light client
sync protocol, and adds a new `nimbus_light_client` executable that uses
this library for syncing. The new executable emits log messages when
new beacon block headers are received, and is integrated into testing.
2022-05-31 12:45:37 +02:00
tersec a3413963a1
update (or for one, remove) 15 CL spec ref URLs (#3671) 2022-05-30 12:24:43 +00:00
tersec d35d408fa7
remove deprecated JSON-RPC server (#3656)
* remove deprecated JSON-RPC server

* keep the command-line options around as no-ops, temporarily

* service -> server; JSON-RPC is still used elsewhere
2022-05-24 07:23:48 +00:00
tersec 5773f71ac0
update engine API URLs to v1.0.0-alpha.9 (#3654) 2022-05-23 15:41:34 +00:00
zah a0a6dd2f63
Add a ncli tool for converting a regular keystore into a distributed one (#3634) 2022-05-17 16:50:49 +03:00
tersec 6e28ce425a
remove --use-jwt-debug (#3619) 2022-05-11 17:50:19 +00:00
tersec 3088da1303
enable JWT by default (#3610) 2022-04-30 12:18:06 +00:00
tersec 09b2f4c237
unhide --jwt-secret flag (#3604) 2022-04-19 11:34:55 +00:00
tersec ab1fac7236
post-merge Bellatrix block proposals (#3570)
* post-merge Bellatrix block proposals

* tolerate running without an Eth1Monitor better

* remove obsolete comment

* use correct empty receipts root

* handle invalid CLI parameters in parseCmdArg overloads
2022-04-14 20:15:34 +00:00
tersec 06da9f4868
remove proposer boosting option (always enabled now) (#3586) 2022-04-13 12:14:00 +03:00
Dustin Brody 6ab5cb1729
fix underlying issue 2022-04-12 20:17:32 +00:00
tersec ff6c581273
keep proposer boosting permanently enabled (#3565) 2022-04-12 12:06:30 +02:00
Jacek Sieka d1a8daec7e
`gnosis-chain` -> `gnosis` (#3577)
* `gnosis-chain` -> `gnosis`

Use same name as LH/Teku throughout

* fixes #3504
* fixes large stack temporary that can cause crashes during genesis
detection
2022-04-08 23:11:37 +03:00
Etan Kissling 509aa6c252
serve light client data on `prater` by default (#3559)
Applies a `prater` testnet specific config default to serve light client
data on that network.
2022-04-01 23:58:06 +02:00
tersec a18b39c9c1
JWT support (#3561) 2022-03-31 14:43:05 +00:00
tersec b37bf8c94b
remove TaintedString (#3546)
* remove TaintedString

* dumpDir is used as a helper function for more specific types of dumpDirs
2022-03-24 21:44:34 +00:00