Commit Graph

364 Commits

Author SHA1 Message Date
tersec e0e7af7aff
MEV boost script for ropsten and sepolia builder network (#3851)
* MEV boost script for ropsten and sepolia builder network

* networkInfo not a global variable
2022-07-12 20:50:12 +03:00
tersec ee27274078
update Kiln book chapter (#3853)
* update Kiln book chapter

* remove reference to kiln branch of nethermind
2022-07-12 07:51:40 +00:00
tersec 35712e0dd0
suggested fee recipient documentation (#3729)
* tentative suggested fee recipient description

* numbered priority list

* add chapter to book
2022-07-07 09:25:51 +00:00
Jacek Sieka ff12c7f9ce
remove web3 url prompt in launcher script (#3810)
* remove web3 url prompt in launcher script

The interactive prompt for web3 has outlived its utility as we now load
url:s from command line params and config files, preventing the prompt
from correctly detecting when it's needed.

Also, after the merge, a JWT secret will (likely) be needed.

* log notice when web3 url is missing
* fix docs to not mention default that doesn't exist
* fix scripts to properly quote arguments
2022-06-28 23:49:52 +03:00
Jacek Sieka 8b6f05a146
vc: initial documentation (#3798)
* vc: initial documentation

bare-bones, but enough to get started

* fixup

* Apply suggestions from code review

* remove prater, link upgrade

* reword
2022-06-27 17:24:55 +03:00
Jacek Sieka b3caacd59d
Doc updates (#3771)
* Doc updates

* initial merge guide
* mark JSON-RPC as removed
* expand on binaries
* recommend `-j4` when building
* cleanups

* style fixes
2022-06-19 10:24:01 +03:00
Jacek Sieka c48ea3cdc8
book updates (#3763)
* systemd: use same service file as package
* systemd: recommend setting up a user
* clean up obsolete / ephemeral notes
* eth1 -> execution, in many places
* clean up docker instructions to match the experience of running the
runner script
* add basic instructions / links for execution clients other than geth
* clear a few more references to JSON-RPC, update to REST
* update hardware requirements with execution client extras
* start talking about merge requirements (this needs a dedicated page)
* note that binaries do exist, where relevant
* list gitcoin under donations
* update syncing docs
* `NOT` is for `NOTICE` in logs
* update rpi hardware recommendations
* prevent automatic doppelganger restarts in systemd service file
2022-06-17 18:28:13 +03:00
zah e8efc0f184
Add support for the Sepolia network (#3762) 2022-06-16 17:11:26 +03:00
tersec cc5f95dbbb
separate non-zero exit code for doppelganger detection (#3728) 2022-06-10 14:53:19 +03:00
tersec 1262e50fa5
document MEV/mergemock test script (#3694) 2022-06-06 16:55:02 +03:00
Jacek Sieka 6a0433cb08
REST parameter defaults (#3689)
* add default BN for VC
* unify "default" beacon node across commands
* clean up validator exit docs
2022-06-01 10:47:52 +00:00
tersec a3413963a1
update (or for one, remove) 15 CL spec ref URLs (#3671) 2022-05-30 12:24:43 +00:00
tersec 92c1e75915
stop using --rpc in run-beacon-node and simulations (#3662) 2022-05-24 16:21:55 +00:00
tersec b3d603f364
more CL spec URL updates to v1.2.0-rc.1 (#3657) 2022-05-24 08:26:35 +00:00
0kok0 bb7662df3a
Security note (#3646) 2022-05-20 18:28:00 +03:00
zah e7ce3cacd0
Add support for the Ropsten beacon chain (#3648) 2022-05-20 18:26:07 +03:00
Nikolay Mitev b3dc9e2c08
Update merge test vector script and documentation (#3638) 2022-05-16 16:12:32 +03:00
Jacek Sieka 011e0ca02f
era file verification (#3605)
* era file verification

Implement and document era file verification

* era file states now come with block applied for easier verification
* clarify conflicting version handling
* document verification requirements
* remove count from name, use start-era, end-root to discover range

* remove obsolete todo

* abstract out block root loading
2022-05-10 03:28:46 +03:00
sacha c8260a4f15
add kiln instructions (do not merge WIP) (#3590)
* add kiln instructions

* update toc

* incorporate dustin's feedback modulo geth
2022-04-18 12:23:26 +02:00
Etan Kissling 745bf474a3
fix typo in e2store docs (#3602)
`BeackBlock` -> `BeaconBlock` typo.
2022-04-16 15:35:30 +00:00
Jacek Sieka f70ff38b53
enable `styleCheck:usages` (#3573)
Some upstream repos still need fixes, but this gets us close enough that
style hints can be enabled by default.

In general, "canonical" spellings are preferred even if they violate
nep-1 - this applies in particular to spec-related stuff like
`genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
Jacek Sieka f20c53be82
Revert "Polish, remove beta warning for validator monitor (#3531)" (#3578)
This reverts commit 540fc5a1af.
2022-04-08 15:05:17 +00:00
Jacek Sieka 540fc5a1af
Polish, remove beta warning for validator monitor (#3531)
Validator monitoring improves logging by giving more specific monitoring
information, and can now be seen as complete.

Previously, logging has focused on "Attestation sent" messages which
carry little informational value when things go wrong, and are overly
aggressive when everything works as expected (sending attestations is
the norm).

* lower "Attestation sent" log to `INFO`
* mark 1.7.0 as the start of the validator monitor feature - previous
versions had significant bugs in totals mode
2022-04-06 19:44:03 +03:00
Jacek Sieka 30eef0a369
Validator monitor polish (#3569)
* lower "Previous epoch attestation missing" to `NOTICE` for easier
filtering
* add delay logging to validator monitor logs
* simplify delay logging code post-`BeaconTime`
2022-04-06 09:23:01 +00:00
Jacek Sieka 6f4fa32c1d
remove bundled pyrmont support (#3568)
The `pyrmont` testnet has been discontinued.

For experiments, it's still possible to run pyrmont nodes by passing a
genesis/config, but this PR removes the bundled `--network:pyrmont`
option.

* update docs
* remove empty docs
* remove obsolete `eth2-stats` page
2022-04-05 12:59:25 +00:00
Jacek Sieka 4207b127f9
era: load blocks and states (#3394)
* era: load blocks and states

Era files contain finalized history and can be thought of as an
alternative source for block and state data that allows clients to avoid
syncing this information from the P2P network - the P2P network is then
used to "top up" the client with the most recent data. They can be
freely shared in the community via whatever means (http, torrent, etc)
and serve as a permanent cold store of consensus data (and, after the
merge, execution data) for history buffs and bean counters alike.

This PR gently introduces support for loading blocks and states in two
cases: block requests from rest/p2p and frontfilling when doing
checkpoint sync.

The era files are used as a secondary source if the information is not
found in the database - compared to the database, there are a few key
differences:

* the database stores the block indexed by block root while the era file
indexes by slot - the former is used only in rest, while the latter is
used both by p2p and rest.
* when loading blocks from era files, the root is no longer trivially
available - if it is needed, it must either be computed (slow) or cached
(messy) - the good news is that for p2p requests, it is not needed
* in era files, "framed" snappy encoding is used while in the database
we store unframed snappy - for p2p2 requests, the latter requires
recompression while the former could avoid it
* front-filling is the process of using era files to replace backfilling
- in theory this front-filling could happen from any block and
front-fills with gaps could also be entertained, but our backfilling
algorithm cannot take advantage of this because there's no (simple) way
to tell it to "skip" a range.
* front-filling, as implemented, is a bit slow (10s to load mainnet): we
load the full BeaconState for every era to grab the roots of the blocks
- it would be better to partially load the state - as such, it would
also be good to be able to partially decompress snappy blobs
* lookups from REST via root are served by first looking up a block
summary in the database, then using the slot to load the block data from
the era file - however, there needs to be an option to create the
summary table from era files to fully support historical queries

To test this, `ncli_db` has an era file exporter: the files it creates
should be placed in an `era` folder next to `db` in the data directory.
What's interesting in particular about this setup is that `db` remains
as the source of truth for security purposes - it stores the latest
synced head root which in turn determines where a node "starts" its
consensus participation - the era directory however can be freely shared
between nodes / people without any (significant) security implications,
assuming the era files are consistent / not broken.

There's lots of future improvements to be had:

* we can drop the in-memory `BlockRef` index almost entirely - at this
point, resident memory usage of Nimbus should drop to a cool 500-600 mb
* we could serve era files via REST trivially: this would drop backfill
times to whatever time it takes to download the files - unlike the
current implementation that downloads block by block, downloading an era
at a time almost entirely cuts out request overhead
* we can "reasonably" recreate detailed state history from almost any
point in time, turning an O(slot) process into O(1) effectively - we'll
still need caches and indices to do this with sufficient efficiency for
the rest api, but at least it cuts the whole process down to minutes
instead of hours, for arbitrary points in time

* CI: ignore failures with Nim-1.6 (temporary)

* test fixes

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2022-03-23 09:58:17 +01:00
Ștefan Talpalaru ea5c052016
enable multithreading by default (10-20% faster sync) (#3493) 2022-03-19 08:59:10 +01:00
tersec 8fbcf29775
update unchanged specs/phase0/p2p-interface.md URL references from v1.1.9 to v1.1.10 (#3510) 2022-03-16 10:40:35 +00:00
Dustin Brody 346407ef1c
running Nimbus on Kiln 2022-03-13 19:39:11 +00:00
Jacek Sieka d0183ccd77
Historical state reindex for trusted node sync (#3452)
When performing trusted node sync, historical access is limited to
states after the checkpoint.

Reindexing restores full historical access by replaying historical
blocks against the state and storing snapshots in the database.

The process can be initiated or resumed at any point in time.
2022-03-11 12:49:47 +00:00
zah 2dd02e54bc
Update the options page in the guide with explanation regarding the format of the config file (#3475) 2022-03-10 17:02:46 +02:00
Zahary Karadjov 13b264d509
Final v22.3.0: Add examples for migrating from JSON-RCP to REST 2022-03-10 16:46:11 +02:00
Zahary Karadjov 7340e7cab9
v22.3.0
* Deprecates the JSON-RPC API
2022-03-07 21:49:12 +02:00
TennisBowling 187eabeeb9
Docs fix (#3397)
* geth is bigger

* Update preparation.md

* Update install.md

* Update start-syncing.md

* Update docs/the_nimbus_book/src/start-syncing.md

Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>

* Update install.md

* Update docs/the_nimbus_book/src/hardware.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

* Update docs/the_nimbus_book/src/install.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

* Update docs/the_nimbus_book/src/install.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

* Update docs/the_nimbus_book/src/install.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

* Update docs/the_nimbus_book/src/start-syncing.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

* Update docs/the_nimbus_book/src/preparation.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

* Update docs/the_nimbus_book/src/preparation.md

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>
Co-authored-by: sacha <sacha.saint-leger@ethereum.org>
2022-03-06 08:40:17 +01:00
tersec 35f712e378
kiln v2 documentation and testing (#3446) 2022-03-05 16:40:58 +02:00
tersec f0ada15dac
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455) 2022-03-02 10:00:21 +00:00
tersec 05bc61b712
add mev-boost RPC test, with docs (#3430)
* bump nim-web3 and add mev-boost RPC test, with docs

* remove trailing space

* use specific commithash
2022-02-24 14:38:31 +01:00
Paweł Bylica af3c65a689
Fix typo "step 3" -> "step 2" in migration.md (#3404) 2022-02-17 14:29:12 +00:00
sacha e81e67fe65
book updates (#3388) 2022-02-14 12:39:10 +01:00
Dustin Brody 3daa52ab87
update docs for geth/kiln 2022-02-11 20:06:06 +00:00
Jeremy Schlatter 47b1870100
update prysm export docs (#3365)
This command was [recently renamed](https://github.com/prysmaticlabs/prysm/pull/9873).
2022-02-09 11:25:12 +01:00
yslcrypto 71d98a03b3 add warning to REST API page 2022-02-08 18:30:59 +01:00
tersec de0d473ea1
docs: don't rely on ncli_db pruneDatabase for reducing storage usage (#3355)
* don't rely on ncli_db pruneDatabase for reducing storage usage

* remove "Running out of storage" section altogether

* Update docs/the_nimbus_book/src/troubleshooting.md

Co-authored-by: sacha <sacha@status.im>

Co-authored-by: sacha <sacha@status.im>
2022-02-03 12:35:34 +00:00
tersec 8e6a920bf4
rename MERGE_FORK_EPOCH to BELLATRIX_FORK_EPOCH (#3350)
* rename MERGE_FORK_EPOCH to BELLATRIX_FORK_EPOCH

* fix REST test rules
2022-02-02 14:06:55 +01:00
Zachinquarantine f5de887df7
Delete Pyrmont docs (#3340)
* Delete pyrmont.md

* Update log-rotate.md

* Update pi-guide.md
2022-02-01 12:05:20 +01:00
sacha 7d731322b2
Book: trusted sync (edits and clarifications) (#3329)
* edits

* more edits and clarifications

* edit

* add clarification on --trusted-node-url

* address feedback

* remove repetition
2022-02-01 12:02:04 +01:00
Jacek Sieka 3df9ffca9f val-mon: remove redundant `_total` suffix from counters
It turns out nim-metrics adds this suffix on its own - it also turns out
some of the names are non-conventional and need follow-up.
2022-01-31 18:51:24 +02:00
tersec 29e2169585
phase 0 & altair beacon chain and altair validator spec URL updates (#3339) 2022-01-29 13:53:31 +00:00
tersec 89ffa8a1a7
spec URL & copyright year update (#3338) 2022-01-29 01:05:39 +00:00
Jacek Sieka 5dd362fc6e
book: add trusted node sync to index (#3326)
* book: add trusted node sync to index

...and some doc updates

* Update docs/the_nimbus_book/src/start-syncing.md

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2022-01-27 10:19:13 +01:00