status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

v0.6.4 Script and instructions for joining Pyrmont; WIP page in the book

This commit is contained in:
Zahary Karadjov 2020-11-18 00:27:08 +02:00
parent 11e1a9e8e8
commit 6b6e9b4d77
No known key found for this signature in database
GPG Key ID: C8936F8A3073D609
6 changed files with 253 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
CHANGELOG.md
View File

@ -1,39 +1,54 @@
Next 2020-11-17 v0.6.4
================= =================
* Avoid pcre dependency New additions:
* Increase quarantine size to 32 and fix parents not being downloaded
consistently * Support for the Pyrmont testnet.
* The PCRE library is no longer necessary for building Nimbus.
* Sensitive files such as keystores are now accessible only to the
user of the beacon node on POSIX systems (the group rights have
been dropped).
We've fixed:
* An issue preventing blocks to be downloaded when the client goes
out of sync.
* Resource leaks that may lead to reduction of network activity due
to a build-up of malfunctioning peer connections.
2020-11-12 v0.6.2 2020-11-12 v0.6.2
================= =================
A bugfix release addressing issues discovered in the Toledo network. A bugfix release addressing issues discovered in the Toledo network.
New additions: New features include:
* Added an estimated syncing time to the beacon node status bar. * GossipSub 1.1
* Added metadata for the upcoming Pyrmont network. This will be a * The beacon node status bar (footer) now contains a
short-lived network with parameters very closely resembling mainnet. time-left-until-synced estimate.
* Switched to version 1.1 of the GossipSub protocol. * A JSON-RPC method `setLogLevel` for dynamically changing the
* Added an JSON-RPC method `setLogLevel` for dynamically changing the
log level of selected components at run-time. log level of selected components at run-time.
Bug fixes: * The ability to launch Nimbus with a partially-synced Geth node.
* Fixed a bug preventing the node from proposing blocks when connected We've fixed:
* A bug preventing the node from proposing blocks when connected
to a web3 provider to a web3 provider
* Fixed an invalid "corrupted database" error message appearing on start-up * An invalid "corrupted database" error message appearing on start-up
* Fixed incorrectly set message-ids in gossip message causing other clients * Incorrectly set message-ids in gossip message causing other clients
to penalise and potentially disconnect our nodes from the network. to penalise and potentially disconnect our nodes from the network.
* Improved the behaviour of Nimbus when paired with a Geth node which is not * An issue occuring when Nimbus is paired with a Geth node
fully synced. that is not fully synced.
2020-11-09 Hope (v0.6.0) 2020-11-09 Hope (v0.6.0)

26
README.md
View File

@ -112,40 +112,34 @@ apt install build-essential git
### Connecting to testnets ### Connecting to testnets
Once the [prerequisites](#prerequisites) are installed you can connect to the [Medalla testnet](https://github.com/goerli/medalla) with the following commands: Once the [prerequisites](#prerequisites) are installed you can connect to the [Pyrmont testnet](https://github.com/protolambda/pyrmont) with the following commands:
```bash ```bash
git clone https://github.com/status-im/nimbus-eth2 git clone https://github.com/status-im/nimbus-eth2
cd nimbus-eth2 cd nimbus-eth2
make medalla # This will build Nimbus and all other dependencies make pyrmont # This will build Nimbus and all other dependencies
# and connect you to Medalla # and connect you to Pyrmont
``` ```
You can also start multiple local nodes, in different terminal windows/tabs, by specifying their numeric IDs: You can also start multiple local nodes, in different terminal windows/tabs, by specifying their numeric IDs:
```bash ```bash
make medalla NODE_ID=0 # the default make pyrmont NODE_ID=0 # the default
make medalla NODE_ID=1 make pyrmont NODE_ID=1
make medalla NODE_ID=2 make pyrmont NODE_ID=2
``` ```
To change the TCP and UDP ports from the default value of 9000: To change the TCP and UDP ports from the default value of 9000:
```bash ```bash
make BASE_PORT=9100 medalla make BASE_PORT=9100 pyrmont
```
If you wish to make a deposit, execute the following command:
```
make medalla-deposit VALIDATORS=2 # The default is just 1 deposit
``` ```
### Getting metrics from a local testnet client ### Getting metrics from a local testnet client
```bash ```bash
# the primitive HTTP server started to serve the metrics is considered insecure # the primitive HTTP server started to serve the metrics is considered insecure
make NIMFLAGS="-d:insecure" medalla make NIMFLAGS="-d:insecure" pyrmont
``` ```
Now visit http://127.0.0.1:8008/metrics to see the raw metrics. You should see a plaintext page that looks something like this: Now visit http://127.0.0.1:8008/metrics to see the raw metrics. You should see a plaintext page that looks something like this:
@ -181,7 +175,7 @@ See [this page](https://status-im.github.io/nimbus-eth2/metrics-pretty-pictures.
To feed the data into a Prometheus instance, run: To feed the data into a Prometheus instance, run:
```bash ```bash
cd build/data/shared_medalla_0 cd build/data/shared_pyrmont_0
prometheus --config.file=./prometheus.yml --storage.tsdb.path=./prometheus prometheus --config.file=./prometheus.yml --storage.tsdb.path=./prometheus
# when starting multiple nodes at the same time, just use the config file from the one with the highest ID # when starting multiple nodes at the same time, just use the config file from the one with the highest ID
``` ```
@ -192,7 +186,7 @@ You can then visualise the data by getting Grafana up and running with the dashb
### Stress-testing the client by limiting the CPU power ### Stress-testing the client by limiting the CPU power
```bash ```bash
make medalla CPU_LIMIT=20 make pyrmont CPU_LIMIT=20
``` ```
The limiting is provided by the cpulimit utility, available on Linux and macOS. The limiting is provided by the cpulimit utility, available on Linux and macOS.

2
beacon_chain/version.nim
View File

@ -9,7 +9,7 @@ when not defined(nimscript):
const const
versionMajor* = 0 versionMajor* = 0
versionMinor* = 6 versionMinor* = 6
versionBuild* = 3 versionBuild* = 4
versionBlob* = "hope" # Single word - ends up in the default graffitti versionBlob* = "hope" # Single word - ends up in the default graffitti
useInsecureFeatures* = defined(insecure) useInsecureFeatures* = defined(insecure)

8
docker/dist/README.md vendored
View File

@ -28,18 +28,18 @@ make dist
No `-march=native` and no metrics support. No `-march=native` and no metrics support.
## Running a Medalla node ## Running a Pyrmont node
With default options: With default options:
```bash ```bash
./run-medalla-beacon-node.sh ./run-pyrmont-beacon-node.sh
``` ```
The script will forward all supplied options to the beacon node executable: The script will forward all supplied options to the beacon node executable:
```bash ```bash
./run-medalla-beacon-node.sh --log-level=DEBUG --tcp-port=9050 ./run-pyrmont-beacon-node.sh --log-level=DEBUG --tcp-port=9050
``` ```
To monitor the Eth1 validator deposit contract, you'll need to pair To monitor the Eth1 validator deposit contract, you'll need to pair
@ -54,7 +54,7 @@ By default, the script will ask you to enter a web3 provider URL interactively,
but this can be bypassed by specifying a websocket `WEB3_URL` environment variable: but this can be bypassed by specifying a websocket `WEB3_URL` environment variable:
```bash ```bash
WEB3_URL="<YOUR_WEB3_URL>" ./run-medalla-beacon-node.sh WEB3_URL="<YOUR_WEB3_URL>" ./run-pyrmont-beacon-node.sh
``` ```
## Running a mainnet node ## Running a mainnet node

201
docs/the_nimbus_book/src/pyrmont.md Normal file
View File

@ -0,0 +1,201 @@
# Become a Pyrmont validator
This chapter will take you through how to become a validator on the eth2 testnet [Pyrmont](https://github.com/protolambda/pyrmont).
If you generated your signing key using the [eth2 launchpad](https://pyrmont.launchpad.ethereum.org/), and wish to import it into the Nimbus client, this page is for you.
> If you haven't created your validator key yet, we recommend you do so using the [launchpad](https://pyrmont.launchpad.ethereum.org/). If you're not sure what the eth2 launchpad is, we recommend reading this [introductory post](https://blog.ethereum.org/2020/07/27/eth2-validator-launchpad/) first.
## Prerequisites
> ⚠️ If this is your first time playing with Nimbus, please make sure you [install our external dependencies](./install.md) first.
This tutorial assumes basic knowledge of the [command line](https://www.learnenough.com/command-line-tutorial/basics#:~:text=Learn%20Enough%20Command%20Line%20to%20Be%20Dangerous%20is%20an%20introduction,broad%20an%20audience%20as%20possible.).
## Validating
To start validating on the `pyrmont` network:
#### 1. Clone the nim beacon chain repository
```
git clone https://github.com/status-im/nimbus-eth2
cd nimbus-eth2
```
#### 2. Build the beacon node
```
make nimbus_beacon_node
```
*Patience... this may take a few minutes.*
#### 3. Import keystore(s)
```
build/nimbus_beacon_node deposits import --data-dir=build/data/shared_pyrmont_0 <YOUR VALIDATOR KEYS DIRECTORY>
```
Replacing `<YOUR VALIDATOR KEYS DIRECTORY>` with the full pathname of your `validator_keys` directory.
> Tip: run `pwd` in your `validator_keys` directory to print the full pathname to the console.
You'll be asked to enter the password you created to encrypt your keystore(s) in the *Generate Keys* section of the Launchpad process. Don't worry, this is entirely normal. Your validator client needs both your keystore(s) and the password encrypting it to import your [keys](https://blog.ethereum.org/2020/05/21/keys/) (since it needs to decrypt the keystore in order to be able to use it to sign on your behalf).
#### 4. Connect to Pyrmont
```
./run-pyrmont-beacon-node.sh
```
You should see that the beacon node has launched with your validator attached:
```
WRN 2020-08-03 16:24:17.950+02:00 Validator not in registry (yet?) topics="beacval" tid=11677993 file=validator_duties.nim:53 pubKey=a9c4df36
INF 2020-08-03 16:24:17.951+02:00 Local validator attached tid=11677993 file=validator_pool.nim:21 pubKey=a9c4df36 validator=a9c4df36
INF 2020-08-03 16:24:17.951+02:00 Local validators attached topics="beacval" tid=11677993 file=validator_duties.nim:61 count=1
INF 2020-08-03 16:24:17.958+02:00 Starting beacon node topics="beacnde" tid=11677993 file=nimbus_beacon_node.nim:875 version="0.5.0 (31b33907)" nim="Nim Compiler Version 1.2.6 [MacOSX: amd64] (bf320ed1)" timeSinceFinalization=81350 head=ebe49843:0 finalizedHead=ebe49843:0 SLOTS_PER_EPOCH=32 SECONDS_PER_SLOT=12 SPEC_VERSION=0.12.2 dataDir=build/data/shared_pyrmont_0 pcs=start_beacon_node
```
> **Note:** when you run `./run-pyrmont-beacon-node.sh`, the beacon node launches with an Infura endpoint supplied by us. This endpoint is passed through the `web3-url` option (which takes as input the url of the web3 server from which you'd like to observe the eth1 chain).
>
> Because Infura caps the requests per endpoint per day to 100k, and all Nimbus nodes use the same Infura endpoint by default, it can happen that our Infura endpoint is overloaded (i.e the requests on a given day reach the 100k limit). If this happens, all requests to Infura using the default endpoint will fail, which means your node will stop processing new deposits.
>
> To pass in your own Infura endpoint, you'll need to run:
>```
> make NODE_PARAMS="--web3-url=<YOUR_WEBSOCKET_ENDPOINT>" pyrmont
>```
> Importantly, the endpoint must be a websocket (`wss`) endpoint, not `https`. If you're not familiar with Infura, we recommend reading through our [Infura guide](./infura-guide), first.
>
> P.S. We are well aware that Infura is less than ideal from a decentralisation perspective. As such we are in the process of changing our default to [Geth](https://geth.ethereum.org/docs/install-and-build/installing-geth) (with Infura as a fallback). For some rough notes on how to use Geth with Nimbus, see [here](https://gist.github.com/onqtam/aaf883d46f4dab1311ca9c160df12fe4) (we will be adding more complete instructions very soon).
> **Tip:** to 🎨 on the [graffitwall](https://pyrmont.beaconcha.in/graffitiwall), pass the graffiti parameter like this:
>```
> ./run-pyrmont-beacon-node.sh --graffiti='<YOUR_GRAFFITI>'
#### 5. Keep an eye on your validator
If you deposited after the [genesis](https://hackmd.io/@benjaminion/genesis) state was decided (August 2nd 1300 UTC), your validators will have been put in a queue based on deposit time, and will slowly be inducted into the validator set after genesis. Getting through the queue may take a few hours or days.
The best way to keep track of your validator's status is [pyrmont.beaconcha.in](https://pyrmont.beaconcha.in) (click on the orange magnifying glass at the very top and paste in its public key).
You can even [create an account](https://pyrmont.beaconcha.in/register) to add alerts and keep track of your validator's [performance](https://pyrmont.beaconcha.in/dashboard).
To keep track of your syncing progress, have a look at the output at the very bottom of the terminal window in which your validator is running. You should see something like:
```
peers: 35 ❯ finalized: ada7228a:8765 ❯ head: b2fe11cd:8767:2 ❯ time: 9900:7 (316807) ❯ sync: wPwwwwwDwwDPwPPPwwww:7:4.0627 (280512)
```
Where:
- `peers` tells you how many peers you're currently connected to (in the above case, 35 peers)
- `finalized` tells you the most recent finalized epoch you've synced to so far (the 8765th epoch)
- `head` tells you the most recent slot you've synced to so far (the 2nd slot of the 8767th epoch)
- `time` tells you the current time since Genesis (the 7th slot of the 9900th epoch -- or equivalently, the 316,807th slot)
- `sync` tells you how fast you're syncing (4.0627 blocks per second), how many blocks you've synced so far (280,512), along with information about 20 sync workers linked to the 20 most performant peers you are currently connected to (represented by a string of letters and a number).
To dig into `sync` a little:
```
sync: <sync worker map>:<number of active workers>:<current syncing speed in blocks/second>
```
The string of letters -- what we call the `sync worker map` (in the above case represented by `wPwwwwwDwwDPwPPPwwww`) represents the status of the sync workers mentioned above, where:
```
s - sleeping (idle),
w - waiting for a peer from PeerPool,
R - requesting blocks from peer
D - downloading blocks from peer
P - processing/verifying blocks
U - updating peer's status information
```
The number following it (in the above case represented by `7`) represents the number of workers that are currently active (i.e not sleeping or waiting for a peer).
#### 6. Keep your validator updated
Finally, makes sure you stay on the lookout for any critical updates to Nimbus. This best way to do so is through the **announcements** channel on our [discord](https://discord.com/invite/XRxWahP).
To update to the latest version, disconnect from pyrmont and run:
```
git pull && make update
```
Once the update is complete, run `make nimbus_beacon_node` and `./run-pyrmont-beacon-node.sh` to reconnect to the network.
Looking forward to seeing you on Pyrmont! 💛
## Key management
Keys are stored in the `build/data/shared_pyrmont_0/` folder, under `secrets` and `validators` - make sure you keep these folders backed up.
The `secrets` folder contains the common secret that gives you access to all your validator keys.
The `validators` folder contains your keystores (encrypted keys). Keystores are used by validators as a method for exchanging keys. For more on keys and keystores, see [here](https://blog.ethereum.org/2020/05/21/keys/).
>**Note:** The Nimbus client will only ever import your signing key -- in any case, if you used the deposit launchpad, this is the only key you should have (you can generate the withdrawal key from your mnemonic when you wish to withdraw).
## Metrics
Metrics are not included in the binary by default - to enable them, use the following options when building the client:
```
make NIMFLAGS="-d:insecure" nimbus_beacon_node
```
You can then browse the metrics by connecting to:
[http://localhost:8008/metrics](http://localhost:8008/metrics)
Make sure this port is protected as the http server used is not considered secure (it should not be used by untrusted peers).
For instructions on how to spin up a beautiful and useful monitoring dashboard for your validator and beacon node, see [this page](./metrics-pretty-pictures.md).
## Advanced options
### Start multiple nodes
You can start multiple local nodes, in different terminal windows/tabs, by specifying numeric IDs:
```
NODE_ID=0 ./run-pyrmont-beacon-node.sh # the default
NODE_ID=1 ./run-pyrmont-beacon-node.sh
NODE_ID=2 ./run-pyrmont-beacon-node.sh
```
### Attach multiple validators to the same beacon node
Simply [import as many keystores as you wish](./pyrmont.md#3-import-keystores) before running `./run-pyrmont-beacon-node.sh`. Nimbus will automagically find your keys and attach your validators. See [key management](./pyrmont.md#key-management) for more information on where we store your keys.
To give you some context, we (the Nimbus team) are currently running 170 validators per beacon node on our AWS instances.
### Node parameters
You can customise your beacon node's parameters using the `NODE_PARAMS` option:
```
make NODE_PARAMS="--tcp-port=9100 --udp-port=9100" pyrmont
```
>**Note:** the above command has exactly the same effect as `make BASE_PORT=9100 pyrmont`
A complete list of the available parameters can be found [here](https://github.com/status-im/nimbus-eth2/blob/devel/beacon_chain/conf.nim#L92-L210) (use a parameter's `name` field to set it).
### Logs
Log files are saved in `build/data/shared_pyrmont_0/`.
### Makefile
If you are comfortable reading [Makefiles](https://en.wikipedia.org/wiki/Makefile#:~:text=A%20makefile%20is%20a%20file,to%20generate%20a%20target%2Fgoal), you can see the commands that `make pyrmont` executes under the hood, [here](https://github.com/status-im/nimbus-eth2/blob/23bec993414df904e9d7ea9d26e65005b981aee0/Makefile#L184-L197).
Some of the provided options (such as `--network=pyrmont`) are essential while others (such as the ones controlling logging, metrics, ports, and the RPC service) are there for convenience.
The Goerli testnet parameters (`$(GOERLI_TESTNETS_PARAMS`), are defined higher up in the Makefile, [here](https://github.com/status-im/nimbus-eth2/blob/23bec993414df904e9d7ea9d26e65005b981aee0/Makefile#L164-L171).

5
run-pyrmont-beacon-node.sh Normal file
View File

@ -0,0 +1,5 @@
#!/bin/bash
cd "$(dirname $0)"
scripts/run-beacon-node.sh nimbus_beacon_node pyrmont $@