Commit Graph

330 Commits

Author SHA1 Message Date
tersec 2b4f987c80
remove pre-v1.4.0 attestation stability subnets (#5402)
* remove pre-v1.4.0 attestation stability subnets

* re-add most of should register stability subnets on attester duties test
2023-09-11 16:03:34 +00:00
Zahary Karadjov c7db7d0f2c
Add CLI options `--genesis-state` and `--genesis-state-url` 2023-09-08 13:10:18 +03:00
zah 2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Eugene Kabanov afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
Eugene Kabanov 757328372a
Dynamic validators set. (#5366)
* Initial commit.

* Fix argument to be optional.

* Adopt options.md.
2023-08-31 15:16:15 +03:00
tersec 8e413d9abb
rename --stop-at-epoch flag to --debug-stop-at-epoch (#5361) 2023-08-30 14:15:19 +03:00
Etan Kissling c211a3849e
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
Tanguy 36413c8f21
Bump libp2p for gossipsub improvements (#5229)
* Bump libp2p

* Add bandwidthEstimate param

* Fix typo and type

* Add config for light client

* Fix unit

* Add debug prefix

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: diegomrsantos <diegomrsantos@gmail.com>
Co-authored-by: Diego <diego@status.im>
2023-08-24 16:04:19 +02:00
tersec f8520c8b1c
use v1.4.0 stability subnets by default (#5276) 2023-08-08 19:01:04 +00:00
tersec 565edfa351
remove unused code (#5158)
* remove unused code

* add copyright header
2023-07-15 16:30:52 +00:00
tersec ba94dc849f
opt-out support for v1.4.0 attestation stability subnets (#5128)
* opt-out support for v1.4.0 attestation stability subnets

* document debug-prefix-flag-policy and remove temporary from flag name
2023-06-30 05:23:15 +00:00
Tanguy 11d18acfeb
Add Yamux experimental support (#5080) 2023-06-23 09:16:30 +02:00
Zahary Karadjov 503cde366c
Switch to --history:prune by default 2023-06-20 02:08:48 +03:00
tersec c27a65129b
add ability to load KZG trusted setup via runtime flag (#5075) 2023-06-14 10:52:00 +02:00
Eugene Kabanov effe8b7f90
VC: Remote BN received block monitoring. (#4856)
* Initial commit with both methods enabled: `poll` and `event`.

* Address review comments.

* Address review comments.
Fix copyright years.

* After bump fixes.
2023-06-08 10:44:32 +02:00
jangko a24f3cde7e
more flexible compile time resource path 2023-06-05 12:52:25 +07:00
Etan Kissling 73b3f40e8d
use correct exception in `parseCmdArg(enr.Record)` (#5009)
* use correct exception in `parseCmdArg(enr.Record)`

`parseCmdArg` is expected to raise `ValueError` but for `enr.Record` we
currently raise `ConfigurationError`. Change to `ValueError` instead.

* lint
2023-05-31 19:07:06 +00:00
tersec 74511f61d1
Use withdrawal credentials as default fee recipient (#4968) 2023-05-17 07:56:37 +03:00
henridf 573228ffa0
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944) 2023-05-15 05:05:12 +00:00
Jacek Sieka b3c6320d56
embed genesis states using incbin (#4905) 2023-05-11 11:11:00 +00:00
henridf 5dfd814588
Load trusted setup (#4870)
* Kzg: Load trusted setup

* scripts/launch_local_testnet.sh: set FIELD_ELEMENTS_PER_BLOB

* Use right setup file for mainnet/minimal

* Force rebuild

* Add comment explaining why build with -f
2023-05-11 11:52:44 +03:00
Etan Kissling 107f733365
make `--trusted-block-root` option visible for CP sync (#4859)
Add `--trusted-block-root` to `--help` for `trustedNodeSync`.
2023-04-26 13:37:58 +02:00
zah 9b2c07c118
Allow exiting multiple validators at once (#4855) 2023-04-25 09:44:01 +03:00
tersec ed7ad56d16
allow for biasing comparisons between engine/builder api bids (#4847) 2023-04-23 22:10:34 +03:00
Etan Kissling e6e4ba9de6
clean up redundant tests and config (#4836)
The consensus-spec-tests already cover the scenarios of our custom test
runner, so the custom tests can be removed. Also cleans up unused config
flags and related unreachable logic.
2023-04-18 21:26:36 +02:00
tersec 228e10f1d9
update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
Etan Kissling cb9e0eed49
allow trusted node sync based on LC trusted block root (#4736)
* allow trusted node sync based on LC trusted block root

Extends `trustedNodeSync` with a new `--trusted-block-root` option that
allows initializing a light client. No `--state-id` must be provided.
The beacon node will then use this light client to obtain the latest
finalized state from the remote server in a trust-minimized fashion.
Note that the provided `--trusted-block-root` should be somewhat recent,
and that security precautions such as comparing the state root against
block explorers is still recommended.

* fix

* workaround for `valueOr` limitations

* reduce magic numbers

* digest len > context len for readability

* move `cstring` conversion to caller

* avoid abbreviations

* `return` codestyle
2023-04-16 06:07:07 +00:00
Miran 0ec7df1042
fix spelling in conf.nim (#4813) 2023-04-12 17:01:29 +00:00
Etan Kissling df7ecd4fe9
update `/eth/v1/debug/fork_choice` format to latest (#4802)
Syncs the `/eth/v1/debug/fork_choice` REST endpoint with latest specs.

- Validity is now reported as tri-state `enum` instead of two `bool`s
- Response includes store's justified and finalized checkpoints
- Additional `ExtraData` field on outer layer (empty for now)

https://github.com/ethereum/beacon-APIs/pull/232
2023-04-10 15:35:14 +00:00
Etan Kissling 450f06566b
accelerate execution layer sync using light client (#4805)
Turn on `--sync-light-client` option by default, now that it has shown
stability in local testnets.
2023-04-10 14:28:46 +00:00
Eugene Kabanov 0ff86e9538
web3signer refactoring and test suite. (#4775)
* Refactor nimbus_signing_node to support Unix signals.

* Fix SN unable to close REST server properly.

* Fix `keys`, `deposit` and `validator_registration` endpoints issues.
Add getValidatorExitSignature() and getDepositMessageSignature() to validator_pool.

* Add /reload endpoint and implementation.
Fix signData to not cancel `timer`.
Fix validator_pool should clear attachedValidators table.

* Diva protocol enhancement implementation.
2023-04-06 16:16:21 +03:00
Zahary Karadjov 865c84cc18
Some documentation for the new 'deposit exit' work-around 2023-03-21 23:06:57 +02:00
cheatfate 596006be08
Workaround for issue #4216. 2023-03-21 23:06:57 +02:00
Zahary Karadjov 4d1b2dd9f5
Merge branch 'stable' into unstable 2023-03-17 17:51:39 +02:00
Zahary Karadjov ee610cbf34
Restore the hidden --web3-force-polling option as a deprecated one 2023-03-14 18:54:15 +02:00
Etan Kissling 5ef1ed5d96
indicate Capella / Shanghai readiness on startup (#4630)
In startup "Launching beacon node" message, log readiness for Capella /
Shanghai as part of config to aid with support requests.
2023-03-13 22:01:15 +01:00
zah 8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
zah 6036f2e7d7
Local sim impovements (#4551)
* Local sim impovements

* Added support for running Capella and EIP-4844 simulations
  by downloading the correct version of Geth.

* Added support for using Nimbus remote signer and Web3Signer.
  Use 2 out of 3 threshold signing configuration in the mainnet
  configuration and regular remote signing in the minimal one.

* The local testnet simulation can now use a payload builder.
  This is currently not activated in CI due to lack of automated
  procedures for installing third-party relays or builders.

  You are adviced to use mergemock for now, but for most realistic
  results, we can create a simple builder based on the nimbus-eth1
  codebase that will be able to propose transactions from the regular
  network mempool.

* Start the simulation from a merged state. This would allow us
  to start removing pre-merge functionality such as the gossip
  subsciption logic. The commit also removes the merge-forcing
  hack installed after the TTD removal.

* Consolidate all the tools used in the local simulation into a
  single `ncli_testnet` binary.
2023-02-23 02:10:07 +00:00
Jakub Sokołowski 1a0b2daa0a
beacon_chain: fix history flag help message (#4650)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-22 14:09:38 +01:00
Eugene Kabanov e91415662b
Keystore cache implementation. (#4372) 2023-02-16 19:25:48 +02:00
zah ff464e49cf
Implement the set of gas_limit end-points in the Keymanager API (#4612)
Fixes #3946
2023-02-15 15:10:31 +00:00
Etan Kissling 09dd64df32
add deployment phase for Capella-ready networks (#4607)
Allow distinguishing Capella-ready networks from non-upgraded networks
based on `CAPELLA_FORK_EPOCH` being set (Zhejiang public testnet).
2023-02-10 14:26:01 +01:00
tersec 63ed5885ab
update engine API URLs to v1.0.0-beta.2 (#4579) 2023-02-01 18:49:36 +00:00
Zahary Karadjov daa99f43c3
Merge branch 'stable' into unstable 2023-01-26 16:47:26 +01:00
Eugene Kabanov 08ed8ad43e
Adopt BN and VC header sizes and requirements, to avoid users confusion with default configuration options. (#4556)
Add comments.
2023-01-26 16:00:10 +01:00
Jacek Sieka fa27ce76a3
vmon: set details mode to default off
The "on" default for validator monitor details incurs a heavy
performance penalty on large-validator setups - this may cause excess
memory usage or slowdowns when metrics are queried - this PR changes the
default to off, as was intended for the 23.1.0 release.
2023-01-26 14:18:52 +01:00
tersec aacc8d702d
remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528) 2023-01-20 14:14:37 +00:00
tersec aea7a0c8b8
remove TTD monitoring (#4486) 2023-01-18 16:01:49 +02:00
Jacek Sieka d8caab500d
Enable validator monitor by default (#4468)
By enabling the validator monitor, more precise information about the
lifecycle of an attestation is logged at the higher `NOTICE` log level
while current `sent` messages are logged at `INF` instead, since they
are less interesting.

In particular, missed attestations and those that vote for the wrong
head are now detected and logged at NOTICE.

In addition to logging, this feature enables rich metrics around
attestation and sync committee performance - by default, validators are
tracked in aggregate but a detailed mode exists as well

This feature has been available since early Nimbus days, but it has now
been tuned and optimised such that it is safe to enable by default, even
for large setups.

* enable automatic validator monitoring by default
* replace `--validator-monitor-totals` flag with
`--validator-monitor-details` - the detailed mode is disabled by default
* lower "sent" log level to `INF` for several messages - in particular
those that are traced by the validator monitor

This is a retake on #3531 which was later reverted in #3578.
2023-01-16 11:28:35 +01:00
Zahary Karadjov b06502bf65
Gnosis const preset 2023-01-13 04:28:29 +02:00