Commit Graph

570 Commits

Author SHA1 Message Date
tersec 40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec ec1d98cc3a
implement deneb block and blob unblinding (#5496) 2023-10-12 03:40:15 +00:00
tersec a4cf203849
fix mainnet bootstrap node comment parsing; fix nimbus guide grammar (#5486) 2023-10-06 19:26:30 +00:00
Miran e9c8f82282
mention Holesky in the book (#5460) 2023-09-28 11:35:42 +01:00
tersec 2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Miran 7eaa8f832b
Fix #2013, add images to the repo (#5438)
* add img for health.md

* add imgs for metrics.md

* add imgs for email.md

* add img for developers.md

* add imgs for grafana/readme.md

---------

Co-authored-by: Etan Kissling <etan@status.im>
2023-09-19 08:00:48 +02:00
Miran 244f930f42
update Holesky docs (#5435)
* update Holesky docs

* Update docs/the_nimbus_book/src/holesky.md

Co-authored-by: Jacek Sieka <jacek@status.im>

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: Etan Kissling <etan@status.im>
2023-09-19 08:00:27 +02:00
Tanguy 4918a4e2e0
Fix direct peers (#5427)
* Fix direct peers

* Support ENRs in DP, use DP in local testnet

* fix docs

* bump libp2p
2023-09-15 18:45:55 +00:00
tersec 5a29ad7e4f
update some consensus-spec URLs to v1.4.0-beta.1 (#5429) 2023-09-14 18:23:59 +00:00
Miran 73ab2e5f39
styling improvements for holesky.md (#5420)
- compact syntax for different options (as in the rest of the guide)
- correct numeration of the list (continuation vs starting from 1)
- add links to useful resources
- a note stating that Prater is deprecated
2023-09-12 13:53:55 +03:00
Zahary Karadjov de8d52eb1a
Add Holesky section to the guide 2023-09-12 03:50:29 +03:00
Zahary Karadjov 88e8a2818a
Add the new options to the Nimbus guide 2023-09-08 14:31:57 +03:00
zah 53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
zah 2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Eugene Kabanov afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
tersec f99bb881f4
use finalized checkpoint as default trusted node documented target (#5382) 2023-09-02 05:10:54 +00:00
Eugene Kabanov 757328372a
Dynamic validators set. (#5366)
* Initial commit.

* Fix argument to be optional.

* Adopt options.md.
2023-08-31 15:16:15 +03:00
tersec 295c3e2e41
rm `state_sim`; capella genesis for `block_sim` (#5331)
* rm state_sim; capella genesis for block_sim

* copyright year
2023-08-21 09:10:15 +00:00
tersec d171303133
update some consensus spec URLs to v1.4.0-beta.1 (#5287) 2023-08-12 10:38:06 +00:00
tersec 85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
tersec 2c424cac87
Merkle is someone's name (#5265) 2023-08-07 10:06:47 +00:00
Miran a15cc10463
update FAQ (#5211)
* update FAQ

* address review comments
2023-08-02 11:53:24 +00:00
dependabot[bot] a18bd40208
Bump pygments from 2.12.0 to 2.15.0 in /docs (#5206)
Bumps [pygments](https://github.com/pygments/pygments) from 2.12.0 to 2.15.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.12.0...2.15.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-20 14:25:45 +00:00
Miran 92da61ce88
update the RPi guide regarding overclocking (#5135) 2023-07-17 10:26:38 +02:00
Miran f2785e3339
add a trusted node sync step to the guide (#5161) 2023-07-07 11:02:58 +02:00
tersec 1e2d3b12c4
update some consensus-specs ref URLs to v1.4.0-beta.0 (#5163) 2023-07-05 16:02:55 +00:00
Miran 038c97fdf3
various small docs improvements (#5059) 2023-07-05 15:43:43 +02:00
tersec ad3b2fb5ed
correct execution client link in docs (#5159) 2023-07-04 10:54:32 +00:00
tersec 0be95571a7
update some consensus spec URLs to v1.4.0-beta.0 (#5150) 2023-06-30 16:12:54 +00:00
tersec ba94dc849f
opt-out support for v1.4.0 attestation stability subnets (#5128)
* opt-out support for v1.4.0 attestation stability subnets

* document debug-prefix-flag-policy and remove temporary from flag name
2023-06-30 05:23:15 +00:00
Miran 283f716593
update required mdbook versions (#5125)
I couldn't install those packages with the versions previously stated.
The new versions install successfully,
and `make publish-book` works without errors.
2023-06-26 16:04:01 +00:00
tersec 614202e30d
automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
Miran cabb2779b4
Make the validating guide self-contained (#5076)
* make the validating guide self-contained

Instead of having links to the three separate how-to guides,
now everything is under one umbrella.

* fix all now-broken links

* Apply suggestions from code review

Co-authored-by: tersec <tersec@users.noreply.github.com>

* address review comments

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-06-20 14:38:10 +00:00
tersec bb63f8e0ec
update docs to reflect default pruning change (#5103) 2023-06-20 14:25:13 +00:00
Zahary Karadjov 883468eb22
Update the guide to match the new default for the --history option 2023-06-20 12:59:36 +03:00
tersec 591c2246d5
update consensus spec URLs to v1.4.0-alpha.3 (#5088) 2023-06-16 16:45:09 +00:00
tersec 939b3002e4
remove removed getDebugChainHeadsV1; fix API links in documentations (#5058) 2023-06-15 09:58:38 +00:00
tersec 788cdb7133
automated v1.4.0-alpha.2 to v1.4.0-alpha.3 consensus spec URL updates (#5065) 2023-06-13 14:03:49 +00:00
tersec 22208836b1
automated v1.4.0-alpha.1 to v1.4.0-alpha.2 consensus spec URL updates (#5056) 2023-06-10 09:56:54 +00:00
tersec 1b77831758
Revert "remove removed Debug/getDebugChainHeads V1 (#4970)" (#5057)
This reverts commit 6ebcbbdf4e.
2023-06-10 08:08:01 +00:00
tersec 6ebcbbdf4e
remove removed Debug/getDebugChainHeads V1 (#4970)
* remove removed Debug/getDebugChainHeads V1

* REST API tests
2023-06-10 08:07:36 +00:00
Miran 73fc1d5a2b
Update "Getting started" part of the book (#4938)
* refactor index.md

* move philosophy.md into index.md

* clean up the 'getting started' topic

move how-to's to the how-to section

* cleaner titles

* move prerequisites to the install.md page

* hardware.md: small refactor

* improve quick-start.md

* refactor eth1.md

* refactor validating guide

* reverse the order of BN and LC in the table

* pi-guide.md: minor edit

* Apply suggestions from code review

Co-authored-by: tersec <tersec@users.noreply.github.com>

* make link texts more concrete

* remove comparison to other clients

* add some more explanations to `build.md`

* change 'days' to 'hours'

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-06-09 23:06:01 +00:00
tersec 30c859e8a4
update some consensus-spec URLs to v1.4.0-alpha.1 (#5035) 2023-06-06 17:09:48 +00:00
tersec f86febc111
update consensus spec URLs to v1.4.0-alpha.1 (#5027) 2023-06-04 10:32:50 +00:00
tersec b076168bf8
State explicitly slashingdb import doesn't destroy existing information (#4998) 2023-05-30 11:17:18 +03:00
tersec 1f535336da
automated v1.3.0 to v1.4.0-alpha.0 consensus spec URL updates (#4996)
* automated v1.3.0 to v1.4.0-alpha.0 consensus spec URL updates

* add copyright year headers
2023-05-26 00:14:28 +02:00
Zahary Karadjov a167424fc0
Version 23.5.1 2023-05-18 20:10:12 +03:00
dependabot[bot] b3c3b9a7d8
Bump pymdown-extensions from 9.5 to 10.0 in /docs (#4963)
Bumps [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) from 9.5 to 10.0.
- [Release notes](https://github.com/facelessuser/pymdown-extensions/releases)
- [Commits](https://github.com/facelessuser/pymdown-extensions/compare/9.5...10.0)

---
updated-dependencies:
- dependency-name: pymdown-extensions
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-16 16:08:41 +00:00
Jacek Sieka 51418a7894
Incremental pruning (#4887)
* Incremental pruning

When turning on pruning the first time the current pruning algorithm
will prune the full database at startup. This delays restart
unnecessarily, since all of the pruned space is not needed at once.

This PR introduces incremental pruning such that we will never prune
more than 32 blocks or the sync speed, whichever is higher.

This mode is expected to become default in a follow-up release.
2023-05-12 13:37:15 +03:00
Etan Kissling 22c298c07b
final portion of trivial v1.3.0 bumps (#4926)
* final portion of trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint
2023-05-11 09:54:29 +00:00
Etan Kissling b04970189b
more v1.3.0 consensus-specs bumps (#4915)
More spec references for unchanged logic bumped to v1.3.0.
2023-05-09 17:20:37 +00:00
zah 5bf9284e62
Initial public version of the Verifying Web3Signer functionality (#4912)
* Allow the list of proved properties for web3signer to be configured
* Document the Web3Signer setups (regular, distributed and verified)
2023-05-09 11:16:43 +03:00
zah 2ae909d9b9
Documentation improvements (#4794) 2023-04-28 00:30:58 +03:00
zah 55ae7d3b00
Version 23.4.0 (#4857)
* Draft release notes for version 23.4.0

* Drop the reference to the Builder API failsafe spec PR
2023-04-25 22:46:38 +03:00
zah 9b2c07c118
Allow exiting multiple validators at once (#4855) 2023-04-25 09:44:01 +03:00
Jacek Sieka 58b93ccbe0
era: Capella+ support (fixes #4752) (#4853)
Post-Capella, historical roots are computed from historical summaries
instead of being directly stored in the beacon state.

Slightly messy to pass both lists around - this is done to avoid
computing the historical root unnecessarily.
2023-04-24 15:26:28 +02:00
tersec ed7ad56d16
allow for biasing comparisons between engine/builder api bids (#4847) 2023-04-23 22:10:34 +03:00
tersec b390911e93
automatic update of v1.3.0-rc.5 consensus spec URLs to v1.3.0 (#4848) 2023-04-21 18:52:43 +00:00
Jacek Sieka addb7bda2f
Merge remote-tracking branch 'origin/stable' into unstable 2023-04-17 13:41:50 +02:00
liftlines 9bc28254cb
Update index.md
Updated the donation address as per announcement.
2023-04-16 17:00:26 +10:00
Miran 0ec7df1042
fix spelling in conf.nim (#4813) 2023-04-12 17:01:29 +00:00
Miran d1318fbe96
Doc fixes, part 2 (#4806)
* part 2 of the initial doc fixes

- spelling fixes
- grammar fixes
- em-dashes should be em-dashes (`—`): double dashes (`--`) are not rendered properly
- reduce overusage of em-dashes, some of those should be separate sentences
- use the correct syntax for notes, tips and warnings
- every sentence is in a separate line (helps with future diffs)
- add missing dots at the end of list items
- fix some lists

* sentences on separate lines in the remaining files
2023-04-11 18:42:35 +03:00
Etan Kissling 450f06566b
accelerate execution layer sync using light client (#4805)
Turn on `--sync-light-client` option by default, now that it has shown
stability in local testnets.
2023-04-10 14:28:46 +00:00
zah 1459189e46
Add a guide for withdrawals (#4790) 2023-04-05 19:23:23 +03:00
Miran 05d99185c9
Various docs fixes (#4789)
- spelling fixes
- use the correct sintax for notes
- em-dashes should be em-dashes (`—`): double dashes (`--`) are not
  rendered properly
- fix a link (was to a .html, should be to .md)
- add missing dots
- every sentence should be on a separate line: it makes future diffs
  much nicer and easier to spot the changes
2023-04-05 16:58:58 +03:00
Zahary Karadjov c9eb89e9e9
Simplify the validator exit docs further 2023-03-22 02:56:55 +02:00
Zahary Karadjov 865c84cc18
Some documentation for the new 'deposit exit' work-around 2023-03-21 23:06:57 +02:00
tersec 2f634c10a4
automated consensus spec URL updating from v1.3.0-rc.4 to rc.5 (#4756) 2023-03-21 00:42:22 +00:00
tersec c9b7ec9809
rm obsolete merge testnet documentation (#4747) 2023-03-20 16:02:04 +01:00
Zahary Karadjov 4d1b2dd9f5
Merge branch 'stable' into unstable 2023-03-17 17:51:39 +02:00
tersec 0c62950de6
more consensus spec v1.3.0-rc.4 URL updates (#4744) 2023-03-17 12:14:08 +00:00
tersec ec77116414
automated consensus spec URL updating from v1.3.0-rc.3 to rc.4 (#4742) 2023-03-17 01:10:31 +00:00
Zahary Karadjov 1a8b182d98
Some tweaks to the release notes 2023-03-15 16:16:51 +02:00
Zahary Karadjov 46f48269ef
Backwards compatible handling of the web3-url parameter in TOML 2023-03-14 17:50:03 +02:00
Etan Kissling 69013d153c
bump light client spec references to `v1.3.0-rc.3` (#4719) 2023-03-11 01:11:51 +00:00
Zahary Karadjov 17c0eeeede
Version 23.3.0 2023-03-11 02:51:44 +02:00
zah 8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
Jacek Sieka daf8ee6c59
docs: add apt repo (#4644)
* docs: add apt repo

* Update docs/the_nimbus_book/src/keep-updated.md

Co-authored-by: tersec <tersec@users.noreply.github.com>

* more package manager integration

* fix admonition

---------

Co-authored-by: zah <zahary@status.im>
Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-03-02 17:22:07 +01:00
tersec 29fb65a9db
automated update of v1.3.0-rc.2 to v1.3.0-rc.3 consensus spec URLs (#4647) 2023-02-21 16:43:21 +00:00
Zahary Karadjov 8202d4a84e
Version 23.2.0 2023-02-16 20:19:22 +02:00
zah ff464e49cf
Implement the set of gas_limit end-points in the Keymanager API (#4612)
Fixes #3946
2023-02-15 15:10:31 +00:00
Jacek Sieka 822c339607
e2store: avoid uint48 (#4625)
In SSZ, `uint32` is used for offsets, effectively limiting the size of
an SSZ entry to 2**32 bytes.

Also, `uint48` isn't a valid SSZ type, so the header was not correctly
defined according to the SSZ spec - the extra 2 bytes are left for
future expansion instead.
2023-02-15 14:51:17 +01:00
tersec 4c845b9749
note 8GB requirement is for building; fix macOS supported version (#4594) 2023-02-06 08:21:11 +01:00
Andy Tudhope caa9eec394
Update hardware.md (#4584)
Due to increased requirements when building beacon nodes.

Though this is annoying, I still think you all are wonderful and magical beings.
2023-02-02 18:14:54 +01:00
tersec 58ed9308d2
automated v1.3.0-rc.1 to v1.3.0-rc.2 consensus spec URL updates (#4568) 2023-01-31 00:26:57 +01:00
tersec 8c2e01da24
consensus spec ref URL updates to v1.3.0-rc.1 (#4534) 2023-01-20 22:33:05 +00:00
tersec 073c544f0c
automated update from v1.3.0-rc.0 to v1.3.0-rc.1 consensus spec URLs (#4517) 2023-01-17 16:10:52 +00:00
Jacek Sieka d8caab500d
Enable validator monitor by default (#4468)
By enabling the validator monitor, more precise information about the
lifecycle of an attestation is logged at the higher `NOTICE` log level
while current `sent` messages are logged at `INF` instead, since they
are less interesting.

In particular, missed attestations and those that vote for the wrong
head are now detected and logged at NOTICE.

In addition to logging, this feature enables rich metrics around
attestation and sync committee performance - by default, validators are
tracked in aggregate but a detailed mode exists as well

This feature has been available since early Nimbus days, but it has now
been tuned and optimised such that it is safe to enable by default, even
for large setups.

* enable automatic validator monitoring by default
* replace `--validator-monitor-totals` flag with
`--validator-monitor-details` - the detailed mode is disabled by default
* lower "sent" log level to `INF` for several messages - in particular
those that are traced by the validator monitor

This is a retake on #3531 which was later reverted in #3578.
2023-01-16 11:28:35 +01:00
Etan Kissling 2324136552
add `LightClientHeader` wrapper (#4481)
The light client data structures were changed to accommodate additional
fields in future forks (e.g., to also hold execution data).

There is a minor change to the JSON serialization, where the `header`
properties are now nested inside a `LightClientHeader`.
The SSZ serialization remains compatible.

See https://github.com/ethereum/consensus-specs/pull/3190
and https://github.com/ethereum/beacon-APIs/pull/287
2023-01-13 16:46:35 +01:00
tersec e28e1aeec8
a few consensus spec ref URL updates (#4483) 2023-01-10 16:14:17 +00:00
tersec 2dd3cd786f
consensus spec ref URL update v1.3.0-{alpha.2,rc.0}; copyright year update (#4477) 2023-01-09 22:44:44 +00:00
Jacek Sieka 0ba9fc4ede
History pruning (fixes #4419) (#4445)
Introduce (optional) pruning of historical data - a pruned node will
continue to answer queries for historical data up to
`MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs, or roughly 5 months, capping
typical database usage at around 60-70gb.

To enable pruning, add `--history=prune` to the command line - on the
first start, old data will be cleared (which may take a while) - after
that, data is pruned continuously.

When pruning an existing database, the database will not shrink -
instead, the freed space is recycled as the node continues to run - to
free up space, perform a trusted node sync with a fresh database.

When switching on archive mode in a pruned node, history is retained
from that point onwards.

History pruning is scheduled to be enabled by default in a future
release.

In this PR, `minimal` mode from #4419 is not implemented meaning
retention periods for states and blocks are always the same - depending
on user demand, a future PR may implement `minimal` as well.
2023-01-07 10:02:15 +00:00
tersec c5d1683f19
spec ref URL & copyright year updates (#4467) 2023-01-06 16:28:46 +00:00
Jacek Sieka 7c2ed5c609
Always-on optimistic mode (#4458)
With https://github.com/status-im/nimbus-eth2/pull/4420 implemented, the
checks that we perform are equivalent to those of a `SYNCING` EL - as
such, we can treat missing EL the same as SYNCING and proceed with an
optimistic sync.

This mode of operation significantly speeds up recovery after an offline
EL event because the CL is already synced and can immediately inform the
EL of the latest head.

It also allows using a beacon node for consensus archival queries
without an execution client.

* deprecate `--optimistic` flag
* log block details on EL error, soften log level because we can now
continue to operate
* `UnviableFork` -> `Invalid` when block hash verification fails -
failed hash verification is not a fork-related block issue
2023-01-04 15:51:14 +00:00
tersec 9e699fd2fc
manual rebase of Bump wheel from 0.37.1 to 0.38.1 in /docs #4446 (#4456) 2023-01-03 18:02:05 +00:00
zah 0be57eec6e
Don't package the nimbus_validator_client in the nimbus-eth2 docker images; Document the new dedicated image (#4441) 2022-12-20 23:32:31 +02:00
zah 94a87da4a2
Documentation for the --with-deposit-snapshot option (#4440) 2022-12-20 23:32:13 +02:00
tersec bb4ea37baa
update EF consensus spec URLs from v1.3.0-alpha.1 to v1.3.0-alpha.2 (#4432) 2022-12-15 12:15:12 +00:00
tersec ad8e682f76
remove redundant justification and finalization tests (#4412) 2022-12-09 22:45:48 +00:00
Jacek Sieka 6e2a02466e
unify bn/vc doppelganger detection (#4398)
* fix REST liveness endpoint responding even when gossip is not enabled
* fix VC exit code on doppelganger hit
* fix activation epoch not being updated correctly on long deposit
queues
* fix activation epoch being set incorrectly when updating validator
* move most implementation logic to `validator_pool`, add tests
* ensure consistent logging between VC and BN
* add docs
2022-12-09 17:05:55 +01:00