Commit Graph

422 Commits

Author SHA1 Message Date
tersec de0d473ea1
docs: don't rely on ncli_db pruneDatabase for reducing storage usage (#3355)
* don't rely on ncli_db pruneDatabase for reducing storage usage

* remove "Running out of storage" section altogether

* Update docs/the_nimbus_book/src/troubleshooting.md

Co-authored-by: sacha <sacha@status.im>

Co-authored-by: sacha <sacha@status.im>
2022-02-03 12:35:34 +00:00
tersec 8e6a920bf4
rename MERGE_FORK_EPOCH to BELLATRIX_FORK_EPOCH (#3350)
* rename MERGE_FORK_EPOCH to BELLATRIX_FORK_EPOCH

* fix REST test rules
2022-02-02 14:06:55 +01:00
Zachinquarantine f5de887df7
Delete Pyrmont docs (#3340)
* Delete pyrmont.md

* Update log-rotate.md

* Update pi-guide.md
2022-02-01 12:05:20 +01:00
sacha 7d731322b2
Book: trusted sync (edits and clarifications) (#3329)
* edits

* more edits and clarifications

* edit

* add clarification on --trusted-node-url

* address feedback

* remove repetition
2022-02-01 12:02:04 +01:00
Jacek Sieka 3df9ffca9f val-mon: remove redundant `_total` suffix from counters
It turns out nim-metrics adds this suffix on its own - it also turns out
some of the names are non-conventional and need follow-up.
2022-01-31 18:51:24 +02:00
tersec 29e2169585
phase 0 & altair beacon chain and altair validator spec URL updates (#3339) 2022-01-29 13:53:31 +00:00
tersec 89ffa8a1a7
spec URL & copyright year update (#3338) 2022-01-29 01:05:39 +00:00
Jacek Sieka 5dd362fc6e
book: add trusted node sync to index (#3326)
* book: add trusted node sync to index

...and some doc updates

* Update docs/the_nimbus_book/src/start-syncing.md

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2022-01-27 10:19:13 +01:00
sacha ed38b187cc
book: migration guide update (#3316)
* rest api: first draft

* minor edit

* add lighthouse export

* add teku

* cp

* cp

* fix merge conflict"

* fix typo

* incorporate feedback

* correct path mistake
2022-01-25 15:16:50 +01:00
sacha e07044754a
Update rest-api.md 2022-01-24 15:06:35 +01:00
sacha b7b517d97e
improve rest api documentation: first draft (#3306)
* rest api: first draft

* minor edit

* Update docs/the_nimbus_book/src/rest-api.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/rest-api.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/rest-api.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* incorporate feedback

Co-authored-by: Jacek Sieka <jacek@status.im>
2022-01-24 14:46:40 +01:00
lbly 57abb07e4e
Update start-syncing.md (#3300)
I found this documentation a little confusing because it says "you must" leading me to think there may be some kind of dire consequences for NOT having it synced before starting the validator (in the context of importing existing keys from another setup, not starting a new validator with a new deposit).  Modifying the language to make the consequences of not having the syncing done clear.
2022-01-20 17:15:09 +01:00
Jacek Sieka 68247f81b3
Trusted node sync (#3209)
* Trusted node sync

Trusted node sync, aka checkpoint sync, allows syncing tyhe chain from a
trusted node instead of relying on a full sync from genesis.

Features include:

* sync from any slot, including the latest finalized slot
* backfill blocks either from the REST api (default) or p2p (#3263)

Future improvements:

* top up blocks between head in database and some other node - this
makes for an efficient backup tool
* recreate historical state to enable historical queries

* fixes

* load genesis from network metadata
* check checkpoint block root against state
* fix invalid block root in rest json decoding
* odds and ends

* retry looking for epoch-boundary checkpoint blocks
2022-01-17 10:27:08 +01:00
sacha df8d744a47
book updates (min viable changes prior to release) (#3283)
* min viable changes

* make clear keymanager api is not ready for mainnet

* Update docs/the_nimbus_book/src/keymanager-api.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/keymanager-api.md

Co-authored-by: zah <zahary@gmail.com>

Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: zah <zahary@gmail.com>
2022-01-14 13:37:21 +01:00
Zahary Karadjov e67773a495
Document the Keymanager API in the book 2022-01-13 20:19:07 +02:00
Jacek Sieka 6f7e0e3393
REST cleanups (#3255)
* REST cleanups

* reject out-of-range committee requests
* print all hex values as lower-case
* allow requesting state information by head state root
* turn `DomainType` into array (follow spec)
* `uint_to_bytesXX` -> `uint_to_bytes` (follow spec)
* fix wrong dependent root in `/eth/v1/validator/duties/proposer/`
* update documentation - `--subscribe-all-subnets` is no longer needed
when using the REST interface with validator clients
* more fixes
* common helpers for dependent block
* remove test rules obsoleted by more strict epoch tests
* fix trailing commas

* Update docs/the_nimbus_book/src/rest-api.md
* Update docs/the_nimbus_book/src/rest-api.md

Co-authored-by: sacha <sacha@status.im>
2022-01-08 22:06:34 +02:00
Jacek Sieka ba99c8fe4f
update era file documentation / impl (#3226)
Overhaul of era files, including documentation and reference
implementations

* store blocks, then state, then slot indices for easy lookup at low
cost
* document era file rationale
* altair+ support in era writer
2022-01-07 11:13:19 +01:00
zah fba1f08a5e
Implement #3129 (Optimized history traversals in the REST API) (#3219)
* Fix REST some rest call signatures and implement a simple API benchmark tool

* Implement #3129 (Optimized history traversals in the REST API)

Other notable changes:

The `updateStateData` procedure in the `blockchain_dag.nim` module is
optimized to not rewind down to the last snapshot state saved in the
database if the supplied input state can be used as a starting point
instead.

* Disallow await in withStateForBlockSlot
2022-01-05 15:49:10 +01:00
Zahary Karadjov 54d0d588b1 Implementation of the Keymanager API (BETA)
https://github.com/ethereum/keymanager-APIs
2022-01-04 18:51:45 +02:00
tersec da017d2ca5
update from phase0/altair v1.1.6 URLs to v1.1.8 spec URLs (#3238) 2022-01-04 03:57:15 +00:00
Jacek Sieka c270ec21e4
Validator monitoring (#2925)
Validator monitoring based on and mostly compatible with the
implementation in Lighthouse - tracks additional logs and metrics for
specified validators so as to stay on top on performance.

The implementation works more or less the following way:
* Validator pubkeys are singled out for monitoring - these can be
running on the node or not
* For every action that the validator takes, we record steps in the
process such as messages being seen on the network or published in the
API
* When the dust settles at the end of an epoch, we report the
information from one epoch before that, which coincides with the
balances being updated - this is a tradeoff between being correct
(waiting for finalization) and providing relevant information in a
timely manner)
2021-12-20 20:20:31 +01:00
Jacek Sieka 03005f48e1
Backfill support for ChainDAG (#3171)
In the ChainDAG, 3 block pointers are kept: genesis, tail and head. This
PR adds one more block pointer: the backfill block which represents the
block that has been backfilled so far.

When doing a checkpoint sync, a random block is given as starting point
- this is the tail block, and we require that the tail block has a
corresponding state.

When backfilling, we end up with blocks without corresponding states,
hence we cannot use `tail` as a backfill pointer - there is no state.

Nonetheless, we need to keep track of where we are in the backfill
process between restarts, such that we can answer GetBeaconBlocksByRange
requests.

This PR adds the basic support for backfill handling - it needs to be
integrated with backfill sync, and the REST API needs to be adjusted to
take advantage of the new backfilled blocks when responding to certain
requests.

Future work will also enable moving the tail in either direction:
* pruning means moving the tail forward in time and removing states
* backwards means recreating past states from genesis, such that
intermediate states are recreated step by step all the way to the tail -
at that point, tail, genesis and backfill will match up.
* backfilling is done when backfill != genesis - later, this will be the
WSS checkpoint instead
2021-12-13 14:36:06 +01:00
Eugene Kabanov b05734f610
Backward sync support for SyncManager. (#3131)
* Unbundle SyncQueue from sync_manager.nim.
Unbundle Peer scores constants to peer_scores.nim.
Add Forward/Backward enum.

* Further improvements and tests.

* Adopt getRewindPoint() and fix MissingParent handler.

* Remove unused procedures.
Refactor `result` usage.
Fix resetWait().

* Add all the tests and fix the issue with rewind point.

* Fix get() issue.

* Fix flaky tests.

* test fixes

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-12-08 22:15:29 +01:00
Jacek Sieka 1a8b7469e3
move quarantine outside of chaindag (#3124)
* move quarantine outside of chaindag

The quarantine has been part of the ChainDAG for the longest time, but
this design has a few issues:

* the function in which blocks are verified and added to the dag becomes
reentrant and therefore difficult to reason about - we're currently
using a stateful flag to work around it
* quarantined blocks bypass the processing queue leading to a processing
stampede
* the quarantine flow is unsuitable for orphaned attestations - these
should also should be quarantined eventually

Instead of processing the quarantine inside ChainDAG, this PR moves
re-queueing to `block_processor` which already is responsible for
dealing with follow-up work when a block is added to the dag

This sets the stage for keeping attestations in the quarantine as well.

Also:

* make `BlockError` `{.pure.}`
* avoid use of `ValidationResult` in block clearance (that's for gossip)
2021-12-06 10:49:01 +01:00
tersec e6921f808f
cleanups, partly from kintsugi branch (#3161)
* cleanups, partly from kintsugi branch

* re-export shortLog(EthBlock) and preserve exception messages in batchVerify and processBatch
2021-12-05 17:32:41 +00:00
tersec cc51f3fd12
v1.1.{5 -> 6} phase 0 and altair spec URL updates (#3157) 2021-12-03 17:40:23 +00:00
sacha a1db321f0a
Update eth1.md
remover warning
2021-12-01 15:23:51 +01:00
tersec 8da2f5307d
remove obsolete amphora interop documentation (#3145) 2021-12-01 14:01:24 +01:00
sacha 335f7c1072
Update infura-guide.md
revert endpoint advice
2021-11-30 18:37:54 +01:00
rasberrylasagna 13280f230c
[FEAT] Updating some infura doc (#3137)
* [FEAT] updating some infura doc

* [FIX] Suggested changes

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>

Co-authored-by: sacha <sacha.saint-leger@ethereum.org>
2021-11-30 12:54:14 +01:00
tersec cc0dbd5bc0
implement terminal-total-difficulty-override; keep kintsugi TTDs consistent (#3118)
* implement terminal-total-difficulty-override; keep TTD consistent for m2 scripts/docs

* use Option[uint64] instead of uint64
2021-11-25 10:53:31 +00:00
tersec 9e395011d9
update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
tersec 39f6a6534e
document how to run merge local testnet with Nethermind (#3110) 2021-11-17 20:45:39 +01:00
Ștefan Talpalaru c673525591
book: add FRITZ!Box UPnP setup (#3106)
* book: add FRITZ!Box UPnP setup

* Update docs/the_nimbus_book/src/networking.md

* Update docs/the_nimbus_book/src/networking.md

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: sacha <sacha@status.im>
2021-11-16 10:46:27 +00:00
sacha 14cb68c72f
Book: upgrade / downgrade page (#3105)
* cp

* cp
2021-11-15 20:07:30 +01:00
tersec c5fa7387f9
update kintsugi docs (#3104) 2021-11-15 17:01:27 +00:00
Jacek Sieka ec650c7fd7
Support starting from altair (#3054)
* Support starting from altair

* hide `finalized-checkpoint-` - they are incomplete and usage may cause
crashes
* remove genesis detection code (broken, obsolete)
* enable starting ChainDAG from altair checkpoints - this is a
prerequisite for checkpoint sync (TODO: backfill)
* tighten checkpoint state conditions
* show error when starting from checkpoint with existing database (not
supported)
* print rest-compatible JSON in ncli/state_sim
* altair/merge support in ncli
* more altair/merge support in ncli_db
* pre-load header to speed up loading
* fix forked block decoding
2021-11-10 13:39:08 +02:00
zah 2c0be052a5
Reduce the logging-related breaking changes (#3070)
* Reduce the logging-related breaking changes

* Don't disable the stdout log when the `--log-file` option is used
* Rename `--log-stdout` to `--log-format` and hide it
* Deprecate the `--log-file` option
2021-11-10 11:02:18 +02:00
sacha 9d82606eba
Update logging.md 2021-11-09 15:38:48 +01:00
sacha db612ae906
Update logging.md
add warning
2021-11-09 15:38:32 +01:00
sacha 87fa67045f
Update logging.md
replace `--log-stdout` with -`-log-format`. remove references to `--log-file`
2021-11-09 15:21:48 +01:00
tersec 48eba59971
manually-verified v1.1.5 spec URL updates (#3068) 2021-11-09 08:54:59 +00:00
tersec 2e868dc2ba
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
sacha 624345bcee
Update logging.md
mainly style edits
2021-11-08 22:05:11 +01:00
tersec 2c8600e746
mass/mechanical update of 1.1.3 phase0 spec URLs to 1.1.4 in markdown (#3059) 2021-11-08 09:26:18 +00:00
Jacek Sieka ea0a191723
Better REST/RPC error messages (#3046)
* Better REST/RPC error messages
* homogenise block logging (root first)
* homegenise message verification pipeline (verify in
`gossip_verification`, act in `eth2_processor`)
* use `subcommitteeIdx` consistently
* log each sent contribution
* fix block_sim
* fix block topic
* don't recalc root on gossip block validation
* move position loop into sync pool
2021-11-05 17:39:47 +02:00
Ștefan Talpalaru b09cd71348 build: show stderr by default
Also added: `-d:cwarnings` and `-d:limitStackUsage`
2021-11-05 16:06:48 +02:00
sacha 122055b65e
cp (#3053) 2021-11-05 11:33:28 +01:00
Jacek Sieka 233d756518
Logging and startup improvements (#3038)
* Logging and startup improvements

Color support for released binaries!

* startup scripts no longer log to file by default - this only affects
source builds - released binaries don't support file logging
* add --log-stdout option to control logging to stdout (colors, json)
* detect tty:s vs redirected logs and log accordingly
* add option to disable log colors at runtime
* simplify several "common" logs, showing the most important information
earlier and more clearly
* remove line numbers / file information / tid - these take up space and
are of little use to end users
  * still enabled in debug builds and tools
* remove `testnet_servers_image` compile-time option
* server images, released binaries and compile-from-source now offer
the same behaviour and features
* fixes https://github.com/status-im/nimbus-eth2/issues/2326
* fixes https://github.com/status-im/nimbus-eth2/issues/1794
* remove instanteneous block speed from sync message, keeping only
average

before:

```
INF 2021-10-28 16:45:59.000+02:00 Slot start                                 topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:884 lastSlot=2384027 wallSlot=2384028 delay=461us84ns peers=0 head=75a10ee5:3348 headEpoch=104 finalized=cd6804ba:3264 finalizedEpoch=102 sync="wwwwwwwwww:0:0.0000:0.0000:00h00m (3348)"
INF 2021-10-28 16:45:59.046+02:00 Slot end                                   topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:821 slot=2384028 nextSlot=2384029 head=75a10ee5:3348 headEpoch=104 finalizedHead=cd6804ba:3264 finalizedEpoch=102 nextAttestationSlot=-1 nextProposalSlot=-1 nextActionWait=n/a
```

after:

```
INF 2021-10-28 22:43:23.033+02:00 Slot start                                 topics="beacnde" slot=2385815 epoch=74556 sync="DDPDDPUDDD:10:5.2258:01h19m (2361088)" peers=37 head=eacd2dae:2361096 finalized=73782:a4751487 delay=33ms687us715ns
INF 2021-10-28 22:43:23.291+02:00 Slot end                                   topics="beacnde" slot=2385815 nextActionWait=n/a nextAttestationSlot=-1 nextProposalSlot=-1 head=eacd2dae:2361096
```

* fix comment

* documentation updates

* mention `--log-file` may be deprecated in the future
* update various docs
2021-11-02 18:06:36 +01:00
sacha 56c6d6b583
update troubleshooting (#3044)
* update troubleshooting

* remove pyrmont references
2021-11-01 17:46:59 +01:00
tersec 36e37bda40
v1.1.3 spec refs URLs (#3036) 2021-10-27 18:40:17 +00:00
tersec 968a5b3bec
some spec refs to v1.1.3 (#3032) 2021-10-26 18:42:48 +00:00
tersec 8307e9c601
mechanical non-merge v1.1.2 to v1.1.3 spec URL updates (#3030) 2021-10-26 16:44:23 +00:00
sacha 07e4dd5f23
edit networking, remove references to pyrmont (#3027) 2021-10-25 19:40:29 +02:00
sacha dc3c23788a
update networking, update hardware requirements, set up health.md (#3020) 2021-10-21 16:18:46 +02:00
sacha c3c8e27011
Network setup update (#2999)
* cp

* cp

* Update docs/the_nimbus_book/src/faq.md

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>

* Update docs/the_nimbus_book/src/faq.md

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* fix links

* new line

* extip edits

* update read logs

* cp

* cp

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
2021-10-21 15:34:46 +02:00
Jacek Sieka 421bf936ff
odds and ends (#3015)
* `allSyncCommittees` => `allSyncSubcommittees`
* simplify `_snappy` topic generation (avoid pointless string copies)
* simplify gossip id generator (avoid pointless string copies)
* avoid redundant syncnet ENR updates
* simplify topic validation (allow only validated topics)
2021-10-21 15:09:19 +02:00
tersec 6cc8757930
update 18 spec URLs to v1.1.2; switch from eth2.0-specs to consensus-specs (#2990) 2021-10-14 06:30:21 +00:00
tersec 2ad1b7366a
update 62 spec URLs to v1.1.2 (#2979) 2021-10-12 10:17:37 +00:00
tersec 0ae736f397
update 67 spec URLs to v1.1.2 (#2977) 2021-10-12 08:09:59 +00:00
tersec bbd51c02ec
replace obsolete Rayonism interop docs with Amphora interop docs (#2974) 2021-10-11 07:06:13 +00:00
sacha 6f1cfc2a8c
add testing apis (#2953) 2021-10-04 18:34:24 +02:00
zah 653fc72b2e
Document that the REST API is out of beta (#2947) 2021-10-04 13:55:02 +02:00
tersec ca4c6b4c5c
update 30 spec URLs to v1.1.0 (#2914) 2021-09-28 14:01:46 +00:00
tersec aec5cf2b1b
update 31 spec reference URLs to v1.1.0 (#2910) 2021-09-28 07:46:06 +00:00
Etan Kissling cc30bf63b4
update gossip_processing and attestation docs (#2860)
The README file explaining gossip_processing, and the attestation_flow
docs were no longer accurate, as attestations and aggregates no longer
go through a queue (pending batching). This patch updates the docs
accordingly. It also improves some grammar and fixes some typos.
2021-09-27 15:11:10 +02:00
sacha 2ec34e6761
Book edits (#2887)
* cp

* cp
2021-09-22 15:52:42 +02:00
Etan Kissling 7b6976d3fd
bump Nim version in developer docs (#2841)
The developer documentation contains a comment mentioning the version of
Nim that the project is tested against. This comment was not updated for
a while (last Nim update in #2508). Bumping now to be in sync once more.
2021-09-01 14:55:54 +02:00
tersec 9725d15a3e
update spec references from eth2.0-specs to consensus-specs and to v1.1.0-beta.2 (#2822) 2021-08-26 10:21:52 +02:00
Zachinquarantine 860897172a
connect-eth2.md: fix small typo (#2804) 2021-08-22 11:24:56 +02:00
Jacek Sieka c3a0e4ec16
API doc fixes
`getSpecPreset` was removed pre-1.0.0 and has never worked - debug calls
were renamed around 1.1.0.
2021-08-16 18:41:13 +02:00
0xmiel 01f78fb93e
update links to joe's guides (#2775) 2021-08-11 15:29:49 +02:00
0xmiel fe96c3e993
add page on security audit (#2774)
* add page on security audit

* incorporate feedback
2021-08-11 15:16:02 +02:00
Jacek Sieka 3d7bee8502
REST API client, JSON-RPC cleanups (#2756)
This refactoring puts the JSON-RPC and REST APIs on more equal footing
by renaming and moving things around, creating a separation between
client and server, and documenting what they are - the aim is to have a
simple-to-use base to start from when developing API clients, as well as
make it easier to navigate the code when looking for the legacy JSON-RPC
interface vs the new REST API.

* move REST client, serialization and supporting types to spec/eth2_apis
* REST stuff now starts with `rest_`, JSON-RPC stuff starts with `rpc_`,
more or less
* simplify imports such that there's a simple module to import for both
server and client
* map REST type and proc names to yaml spec more closely - in
particular, reuse operation and type names in `rest_types` to make
comparisons against spec more easy
* cleaner separation between client and server modules - modules common
between server and client such as `rest_types` and serialization move to
the spec folder - this allows the client to be built with less knowledge
about server internals
2021-08-03 17:17:11 +02:00
zah d9f2a91374
Remove the obsolete testnet0/1 scripts (#2727)
Also fixes `make eth2_network_simulation`
2021-07-16 13:02:27 +03:00
0xmiel bb0a086be3
book updates (#2721)
* cp

* rename API to JSON-RPC API

* rename api to json-rpc api, add rest api page

* remove extra emacs generated file

* remove extra emacs generated file
2021-07-14 13:14:39 +02:00
zah eb2dc5cbbb
Implement the new Altair req/resp protocols (#2676)
* Implement the new Altair req/resp protocols

Also fixes the altair message-id computation by providing the correct
forkdigest prefix in `isAltairTopic`.

Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-07 12:09:47 +03:00
0xmiel 3c7d10cd2a
misc book updates (#2702) 2021-07-05 16:17:35 +02:00
Ștefan Talpalaru c0bbdb0f9f
bin dist related docs (#2684)
* bin dist related docs

* Update docker/README.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docker/README.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docker/README.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/binaries.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/distribution_internals.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/distribution_internals.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/binaries.md

Co-authored-by: 0xmiel <sacha@status.im>

Co-authored-by: 0xmiel <sacha@status.im>
2021-07-05 12:31:25 +02:00
0xmiel 72544f06c0
Update FAQ + minor edits (#2678) 2021-06-25 11:52:06 +02:00
0xmiel 42738848f3
Update keep-updated.md
fix typo
2021-06-22 12:05:57 +02:00
0xmiel fed0a1dc00
update keep updated page (#2665) 2021-06-21 22:50:26 +02:00
0xmiel 96c489bb3d
Faq update + misc edits (#2653)
* update faq with permissions help, link in migration guide and importing keys

* add link to Git for Windows

* update windows install
2021-06-16 11:55:58 +02:00
0xmiel f2588be9ab
update faq and add example to systemd (#2645) 2021-06-11 13:56:15 +02:00
0xmiel 042600a134
Document ncli_db validatorPerf (#2638)
* add work done so far on ncli_db

* remove file

* Monitor attestation performance: tidy up

* remove test.csv

* tersec's feedback
2021-06-11 11:27:09 +02:00
0xmiel 09a459b1d7
Book updates end may (#2629)
* break up long commands, geth > infura

* fill out profitability page

* fill in 'increase privacy'

* update systemd

* add tersec's restarting loop

* update privacy page

* consolidate eth1 instructions

* tidy up eth1.md

* fill out health.md

* rm contribute-network.md from toc

* edit eth1.md

* incorporate review feedback modulo profits.md

* update health.md

* remove script to monitor logs
2021-06-03 14:43:20 +02:00
tersec ea9ceb693a
update ChainDAG.effective_balance() to use StateData; rm ChainDAG.getBlockByPreciseSlot() (#2622)
* update ChainDAG.effective_balance() to use StateData; rm unused ChainDAG.getBlockByPreciseSlot()

* update get_effective_balances to avoid god object; avoid most memory allocation in Altair epoch reward and penalty processing
2021-06-01 12:40:13 +00:00
Johann Bauer c01357c47f
Book: Add instructions on how to download precompiled release (#2600) 2021-05-26 10:43:06 +02:00
0xmiel b7db2f0a29
Book edits (#2597)
* simplify beacon node quickstart

* remove extra emacs generated file

* focus sync info around slot start message

* sync edits

* log rotate edits + clarifications

* remove validator keys folder
2021-05-24 16:03:45 +02:00
yslcrypto 038102cb92 update toc 2021-05-22 11:24:03 +02:00
yslcrypto b400243bda refocus guide around prater 2021-05-22 11:13:27 +02:00
yslcrypto 7088ed8f89 update book toc 2021-05-21 14:54:36 +02:00
0xmiel 4131e0941f
Book updates may 21 (#2590)
* update deposit page

* edit keys.md

* edit connect-eth2.md

* remove extra emacs generated file

* update run a validator guide

* update system requirements

* update start-syncing.md
2021-05-21 14:49:12 +02:00
Johann Bauer 14c258db8b
Fix typo in book (#2580) 2021-05-20 12:41:33 +02:00
yslcrypto 5bf5804bc1 update faq 2021-05-19 16:29:57 +02:00
yslcrypto 8ca2329a20 minor restructure 2021-05-19 16:26:39 +02:00
yslcrypto 62a98e9c6d minor restructure 2021-05-19 16:21:05 +02:00
TennisBowling d27c7b1f53
Add clarity to docs surrounding command line arguments, suggest data-dir when syncing, and polishing up for mainnet (#2527)
* mainnet

* add clarity to command line rules

* add datadir suggestion when syncing

* remind to pass with all calls

* noCommand added into troubleshooting

* review

Co-authored-by: 0xmiel <sacha.saint-leger@ethereum.org>

* review

Co-authored-by: 0xmiel <sacha.saint-leger@ethereum.org>

* review

Co-authored-by: 0xmiel <sacha.saint-leger@ethereum.org>

* review

Co-authored-by: 0xmiel <sacha.saint-leger@ethereum.org>

* typo

* /lib/systemd/system/ lets you enable on startup

Co-authored-by: 0xmiel <sacha.saint-leger@ethereum.org>
Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>
Co-authored-by: 0xmiel <sacha@status.im>
2021-05-19 16:17:07 +02:00
0xmiel 199ebc995b
restructure, merge, simplify (#2581) 2021-05-19 16:12:25 +02:00
yslcrypto 07533f41ba update binaries page 2021-05-19 12:04:10 +02:00
yslcrypto ed2e1e1c6b update docker page 2021-05-19 11:58:48 +02:00
0xmiel 3fe589a511
Update migration-options.md 2021-05-18 20:18:17 +02:00
0xmiel 80e11ac002
Update migration-options.md 2021-05-18 20:17:41 +02:00
0xmiel 5eba5b530d
Update migration-options.md 2021-05-18 20:15:39 +02:00
0xmiel 2d3b9e4c07
Update migration-options.md
Edits
2021-05-18 18:53:59 +02:00
0xmiel faf07737de
Sacha book updates (ready for review) (#2552)
* add doppelganger detection explanation to FAQ and link to Joe's guide from our Pi guide

* Edit CLI options page

* cp

* merge with ith unstable

* prysm migration guide: add steps until import slashing db

* update systemd

* update troubleshooting

* update pi guide

* metrics: replace winners with community

* update migration guide

* cp

* progress with guides, various edits
2021-05-18 18:42:51 +02:00
tersec 290b889ce6
non-intrusive, novel portions of merge (#2535) 2021-05-04 11:54:19 +00:00
yslcrypto 4cdbc17e39 remove references to devel 2021-04-28 16:06:00 +02:00
0xmiel 7af4f7c665
Guide updates (#2525)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* update binaries page

* rm extra emacs generated files

* remove extra emacs generated file

* fix windows typ

* fix eth1 title

* page on adding additional validator
2021-04-28 15:55:43 +02:00
Sacha Saint-Leger 7e31798649
Nimbus guide simplifications (#2516)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* simplify web3 provider instructions

* add donation address
2021-04-22 13:12:24 +02:00
Sacha Saint-Leger bf3305fa65
Nimbus guide: add database backup page (#2505)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* add page on backing up database
2021-04-14 17:00:47 +02:00
Sacha Saint-Leger f0f6b6869f
Nimbus guide: updates for new release (#2504)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* backupadd  web3 provider pader page

* add checksums page

* update grafana: dashboard challenge recommendations
2021-04-14 16:48:11 +02:00
Mamy Ratsimbazafy b6a3c892d7
Cpu architecture optimization documentation (#2483)
* x86 features

* Update docs/cpu_features.md [skip CI]

Co-authored-by: tersec <tersec@users.noreply.github.com>

* Update docs/cpu_features.md [skip CI]

Co-authored-by: tersec <tersec@users.noreply.github.com>

* less space [skip CI]

* Add ARMv8 [skip ci]

Co-authored-by: tersec <tersec@users.noreply.github.com>
2021-04-08 15:48:43 +02:00
Jacek Sieka f821bc878e
Remove `-d:insecure` compile option (#2468)
With metrics running on top of chronos, the metrics server no longer
needs to be compiled in conditionally - it remains disabled by default.
2021-04-01 14:44:11 +02:00
Jacek Sieka 8f441588cd
Add a few help defaults, format all of them the same (#2284) (#2467)
No support in confutils, so so this code simply hardcodes it
2021-04-01 12:28:57 +00:00
Sacha Saint-Leger 981e25a99b
Nimbus book: update homepage (#2466)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* Update homepage
2021-03-31 14:18:20 +02:00
Sacha Saint-Leger 0355c27194
Nimbus book: add Prater page (#2465)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* add prater testnet page
2021-03-31 13:59:12 +02:00
Zahary Karadjov 1af07ad124
Simple RPC for modifying the Graffiti bytes 2021-03-26 19:42:20 +02:00
Sacha Saint-Leger 5362d966ad
Nimbus guide: Rocket Pool edits (#2446)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* update joe's guide

* incorporate pineapple's feedback
2021-03-22 17:57:22 +01:00
Ștefan Talpalaru b2c4441834
book: document "logrotate" (#2390)
* book: document "logrotate"

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* rewording

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

Co-authored-by: Sacha Saint-Leger <sacha@status.im>
2021-03-22 16:40:26 +01:00
Mentor Palokaj 895fe4baf1
Make it clear that the default port for ETH2 is 9000 udp/tcp (#2421)
* Add default port info

* Add default port info

* Edit CLI and docs to reflect 9000 port

* Apply suggestions from code review

Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>
2021-03-22 08:22:56 +01:00
Sacha Saint-Leger 756c82671d
Update rp-quick-start.md 2021-03-17 17:12:32 +01:00
Sacha Saint-Leger b9f3fc4ceb
Rocket Pool guide (#2425)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* Rocket Pool guide
2021-03-17 15:33:28 +01:00
Jacek Sieka aabdd34704
e2store: add era format (#2382)
Era files contain 8192 blocks and a state corresponding to the length of
the array holding block roots in the state, meaning that each block is
verifiable using the pubkeys and block roots from the state. Of course,
one would need to know the root of the state as well, which is available
in the first block of the _next_ file - or known from outside.

This PR also adds an implementation to write e2s, e2i and era files, as
well as a python script to inspect them.

All in all, the format is very similar to what goes on in the network
requests meaning it can trivially serve as a backing format for serving
said requests.

Mainnet, up to the first 671k slots, take up 3.5gb - in each era file,
the BeaconState contributes about 9mb at current validator set sizes, up
from ~3mb in the early blocks, for a grand total of ~558mb for the 82 eras
tested - this overhead could potentially be calculated but one would lose
the ability to verify individual blocks (eras could still be verified using
historical roots).

```
-rw-rw-r--. 1 arnetheduck arnetheduck   16  5 mar 11.47 ethereum2-mainnet-00000000-00000001.e2i
-rw-rw-r--. 1 arnetheduck arnetheduck 1,8M  5 mar 11.47 ethereum2-mainnet-00000000-00000001.e2s
-rw-rw-r--. 1 arnetheduck arnetheduck  65K  5 mar 11.47 ethereum2-mainnet-00000001-00000001.e2i
-rw-rw-r--. 1 arnetheduck arnetheduck  18M  5 mar 11.47 ethereum2-mainnet-00000001-00000001.e2s
...
-rw-rw-r--. 1 arnetheduck arnetheduck  65K  5 mar 11.52 ethereum2-mainnet-00000051-00000001.e2i
-rw-rw-r--. 1 arnetheduck arnetheduck  68M  5 mar 11.52 ethereum2-mainnet-00000051-00000001.e2s
-rw-rw-r--. 1 arnetheduck arnetheduck  61K  5 mar 11.11 ethereum2-mainnet-00000052-00000001.e2i
-rw-rw-r--. 1 arnetheduck arnetheduck  62M  5 mar 11.11 ethereum2-mainnet-00000052-00000001.e2s
```
2021-03-15 11:31:39 +01:00
tersec dfc3322fe2
last few v1.0.0 spec refs to v1.0.1 (#2404) 2021-03-13 20:51:39 +00:00
Jacek Sieka 74a75d05a0
update doc links 2021-03-10 09:47:42 +01:00
Afr Schoe c6f94ce165
docs: add link to keep-updated in build, fix #2379 (#2380) 2021-03-05 09:53:55 +01:00
Mamy Ratsimbazafy 5d7f9c3a04
Consensus object pools [reorg 4/5] (#2374)
* Add documentation

* make test doesn't try to build the beacon node :/
2021-03-04 10:13:44 +01:00
Mamy Ratsimbazafy 3276dfc683
Consolidate modules by areas [part 1] (#2365)
* Move sync in subfolder

* move validator related thingies in validators

* fix binary builds

* update bounds comment [skip ci]
2021-03-02 11:27:45 +01:00
tersec 28e9568d3c
update docs to use v1.0.1 spec ref URLs (#2368) 2021-03-02 09:32:07 +01:00
Mamy Ratsimbazafy 8dcea299d0
Migrate audit tasks from Notion (#2363) 2021-03-01 11:22:35 +01:00
Mamy Ratsimbazafy 08f063aba9
[RFC - Doc] State of Nimbus block & attestation flows (#2351)
* Expand documentation on block flow [skip ci]

* address review comments [skip ci]

* Update with GossipFlow out [skip ci]

* LocalBlockProposer -> LocalValidatorDuties +  WeakSubjectivitySync

* First outline of attestation flow

* finish up prose
2021-03-01 11:22:16 +01:00
Sacha Saint-Leger 37838db82e
docs readme update (#2346)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* update docs readme
2021-02-19 15:59:40 +01:00
Sacha Saint-Leger 3d25f0db01
book updates (#2345)
* edit log rotation page

* update command line options

* add github edit button
2021-02-19 12:17:48 +01:00
Jacek Sieka ff94e1a212
Document log rotation (#2327)
* Document log rotation

* phrasing updates

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2021-02-18 07:57:41 +01:00
Ștefan Talpalaru 4bb90351f6
books: sanity checks and docs (#2320) 2021-02-15 08:33:49 +01:00
Sacha Saint-Leger 855c3e849d
update intro and binary distribution edits (#2306)
* update intro and binary distribution edits

* stefan's suggestions
2021-02-10 17:29:48 +01:00
yslcrypto 237453ec45 fix api error 2021-02-08 15:00:21 +01:00
Ștefan Talpalaru 979bb39b41 docs: binary distribution internals 2021-02-08 14:53:12 +02:00
yslcrypto 6528707e44 add local simulations to the for developers page 2021-01-27 20:38:16 +01:00
yslcrypto fa07c0f0a1 link to quickstart guide in intro 2021-01-27 19:32:04 +01:00
yslcrypto 3016815860 update quickstart 2021-01-27 19:26:08 +01:00
Ștefan Talpalaru 20435880db CI: build and publish a Docker image for end-users
Also disable the log file and log colours for distribution binaries, to
avoid duplicate logs.
2021-01-26 18:52:59 +02:00
Mamy Ratsimbazafy 70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250)
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
2021-01-25 20:45:48 +02:00
Jakub Sokołowski 095b191aee
add cname file to docs to fix domain issues (#2260)
mdbook has an option called `cname`:
https://rust-lang.github.io/mdBook/format/config.html#html-renderer-options

But it appears to be only included starting from `0.4.3`:
https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#mdbook-043

Which means this will be more robust, if uglier.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 17:57:55 +01:00
yslcrypto 173a2fe018 update intro and metrics tutorial 2021-01-22 16:53:49 +01:00
Sacha Saint-Leger 67cce36f06
fix metrics guide (#2234) 2021-01-13 14:21:00 +01:00
Sacha Saint-Leger 37c3c3877c
add troubleshooting section on time (#2232) 2021-01-13 11:41:47 +01:00
Nikola Ristić 3a59b76a01
Update broken repo link (#2208) 2020-12-30 09:55:20 +01:00
Sacha Saint-Leger be74df70e0
Update systemd page: change metrics default and add note (#2204)
* change default and add note on metrics to systemd page

* add link to metrics guide from systemd page
2020-12-21 17:59:20 +01:00