Commit Graph

150 Commits

Author SHA1 Message Date
Viktor Kirilov d9f9949ef0 use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process 2020-09-02 12:47:00 +03:00
Viktor Kirilov 65d7787b1e 50/50 bn/vc split for the validator keys ON by default for the testnet scripts 2020-09-01 16:39:07 +03:00
tersec af3355e0f8
create local testnet mode for eth2_network (#1494) 2020-08-12 14:16:59 +00:00
Viktor Kirilov 5bbeb38f2d fixes the BN/VC communication - properly getting the attestation duties & also fixed start.sh 2020-08-06 15:29:05 +03:00
protolambda e90c5440e8 make eth1 distance runtime configurable 2020-08-06 14:49:58 +03:00
Zahary Karadjov 4b8ebb5d71 Correct instructions in the README for running prometheus on Medalla 2020-08-05 19:28:35 +03:00
Ștefan Talpalaru fa9f35e148
Jenkins: run local testnet test on macOS (#1391) 2020-07-29 14:08:27 +02:00
Zahary Karadjov f4c19e303a Non-interactive generation of keystores in the local sim 2020-07-28 07:36:25 +03:00
Zahary Karadjov 40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
Ștefan Talpalaru e9193fc9da
eth2_network_simulation: fixes (#1368)
- delete "tests/simulation/{data,validators}" by default, because old
  validator keys can lead to a crash with a cryptic error message
- actually start the Prometheus daemon on `make eth2_network_simulation`
- kill any running Prometheus daemon on exit
- fix some shell script syntax incompatible with Bash
2020-07-23 17:58:54 +02:00
Viktor Kirilov 4767eba465 - properly distributing the keys between BNs & VCs - fixing finalization - will switch to ON by default as soon as finalization becomes overall stable
- added a new altona-vc Makefile target which uses a separate VC process
2020-07-15 16:18:21 +03:00
Zahary Karadjov 540b2828b2
Adapt the local sim scripts to use the new run-time presets 2020-07-10 01:08:54 +03:00
Zahary Karadjov 302cef5e90 Attach all local validators before starting the networking 2020-06-29 18:07:41 +03:00
Zahary Karadjov 8140b4458c Working local sim with WAIT_GENESIS=yes 2020-06-29 02:18:48 +03:00
Zahary Karadjov 9cd8a6c730 Working local sim (without waiting genesis) 2020-06-29 02:18:48 +03:00
Zahary Karadjov 8288b568d2
Fix more Tmux issues disovered when testing on a fresh machine [skip ci] 2020-06-25 15:58:13 +03:00
Zahary Karadjov b67a506b3f
Fixes for local sim on macOS [skip ci] 2020-06-25 12:42:30 +03:00
Zahary Karadjov eb48c05b7f Don't assign validator keys to the bootstrap node; Ganache mode for launch_local_testnet 2020-06-23 07:59:35 +00:00
Zahary Karadjov 7211a99086 Smooth out the differences between Ganache and Infura; Working local sim and Altona target 2020-06-22 17:30:04 +03:00
Zahary Karadjov e65c1f49ce Allow Ganache to bundle more deposits in a single block 2020-06-22 17:30:04 +03:00
Zahary Karadjov ffb613d3c6 Usability enhancements for Tmux
* Press 'q' to quit Tmux
* In case of errors, Tmux will wait for a keypress before closing a pane
2020-06-22 17:30:04 +03:00
Zahary Karadjov 01ea66f24e Some documentation and better defaults 2020-06-22 17:30:04 +03:00
Zahary Karadjov 3d9aff5d23 Fix some logical typos introduced in the rebase 2020-06-22 17:30:04 +03:00
Zahary Karadjov 1def383ad1 Reword the Tmux setup to handle all simulation scenarios in more visible way 2020-06-22 17:30:04 +03:00
Zahary Karadjov e9d68e2f7b Add simulation mode that bootstraps from an Eth1 ganache instance 2020-06-22 17:30:04 +03:00
Zahary Karadjov 7d3160b0b5 Local sim now includes a prometheus server, a Ganache instance and a simulation dashboard server (in Tmux mode) 2020-06-22 17:30:04 +03:00
Viktor Kirilov 72dfe7f578 - updated the validator shell script after the keystore changes
- better logging & retrying requests on the VC side if the BN fails for some reason
- VC now fetches the attestation duties 1 epoch in advance - in the future it will tell the BN to subscribe to the appropriate attestation topics in advance based on that info
- a bunch of other code cleanup & fixes such as better naming for consoles when using multitail, etc.

reviewed in PR #1184 - proper review of the API & VC are pending
2020-06-19 12:21:22 +03:00
tersec ee9f4a2e3f
remove skipMerkleValidation and skipBlockParentRootValidation (#1197) 2020-06-18 07:56:47 +02:00
Ștefan Talpalaru d9b494dff6
run_node.sh: macOS fix 2020-06-13 02:50:35 +02:00
Ștefan Talpalaru 33d55fac08
eth2_network_simulation: mkdir + cosmetic changes 2020-06-12 16:37:36 +02:00
Zahary Karadjov a8113cf2bc Restore the local sim to a working state 2020-06-11 17:40:08 +03:00
Zahary Karadjov 17343442ea Implement more of the KeyStore spec and integrate it in the beacon node 2020-06-11 17:40:08 +03:00
Ștefan Talpalaru e2025c5752
Prometheus & Grafana refactoring
- moved "process_dashboard.nim" in "tools/"
- README: made Witti the documented testnet and added instructions for
  getting metrics out of the local node
- moved Prometheus config file generation in its own script
- the static Grafana dashboard definition now covers all nodes, using
  a variable; only the remote testnet dashboards need to be dynamically
  generated
- "launch_local_testnet.sh" no longer needs a "--grafana" option
2020-06-10 19:55:26 +02:00
Viktor Kirilov 3bae40ae91 - extracted the commands to run a VC into a separate run_validator.sh script
- each BN gets half of its previous validators as inProcess and the other half goes to the respective VC for that BN - using separate data dirs where the keys are copied
    - also removed a few command line options which are no longer necessary
- block proposals originating from a VC are propagated from one BN to the rest properly
- other cleanup & moving code back to  since it is no longer used elsewhere
2020-06-10 13:50:50 +03:00
Ștefan Talpalaru fdf1a75956
Grafana: container regex [skip ci] 2020-06-07 16:13:15 +02:00
Ștefan Talpalaru 7c7b6fb361
Grafana: only show one node per page [skip ci]
by using Grafana variables to get dropdown menus
2020-06-07 01:42:54 +02:00
Zahary Karadjov 0c78fc39e7
Use the latest LibP2P 2020-06-05 19:34:12 +03:00
Ștefan Talpalaru 3a6a9f8135 generate Grafana dashboards for remote testnet nodes 2020-06-03 12:44:16 +03:00
Viktor Kirilov cefd525ab3 more work on the BN/VC split
- fixed comments from the last review
- getting more data VIA RPC, moved some code back into the BN only
- attestation duties being requested as well
2020-05-28 13:01:42 +03:00
Viktor Kirilov 8760494c72 first batch of work towards the VC/BN split:
- we have a new binary which connects via RPC to the respective BN and has an internal clock - waking it up on every slot
- the BN has a new option called --external-validators and currently in order to have the VC binaries to run we need to pass EXTERNAL_VALIDATORS=yes to make
- factored some code out of beacon_node.nim for easier reuse in validator_api.nim and validator_client.nim
- the VC loads its associated private keys from the datadir for its BN
- most of the validator API calls have been implemented as a stub.
- the VC polls its BN at the start of each epoch - getting a list of all active validators for the current epoch - and then continues to request blocks and sign them with its appropriate validators when necessary
2020-05-25 16:23:15 +03:00
Ștefan Talpalaru c4462af4ab
beacon_node: graceful shutdown (#1033)
* beacon_node: graceful shutdown

* separate BeaconNodeStatus and BeaconNode instances
2020-05-19 20:57:35 +02:00
Ștefan Talpalaru 9c37c7ba26
work around Prometheus quirks [skip ci] 2020-05-12 18:36:43 +02:00
Ștefan Talpalaru d7b8de9c5e
Grafana dashboard: change time format [skip ci] 2020-05-11 22:08:25 +02:00
Ștefan Talpalaru 2d78523423
fix Grafana dashboards (#1006)
* fix Grafana dashboards

and refactor a couple of scripts

* start.sh: allow running standalone script
2020-05-11 18:35:53 +00:00
Ștefan Talpalaru a7a50824a1
more metrics (#1004) 2020-05-11 06:25:49 +00:00
Ștefan Talpalaru fa663be029
launch_local_testnet.sh: get the metrics out with "--grafana" 2020-05-06 00:02:39 +02:00
Ștefan Talpalaru 0289db0ff5
eth2_network_simulation: more Grafana metrics shown [skip ci] 2020-05-05 23:04:38 +02:00
Ștefan Talpalaru 784c5e3d73
Jenkins: add timeouts to testnet finalisation tests (#947)
* Jenkins: add timeouts to testnet finalisation tests

- add nicer child process management to "launch_local_testnet.sh"
- correct a comment in "start.sh"

* the Jenkins worker's version of "timeout" doesn't support "-v"
2020-04-30 13:59:57 +00:00
Zahary Karadjov 740b76d152 Spec-compliant implementation of Eth1 monitoring; Eth1-enabled local sim
BEWARE! This commit will trigger a stack overflow during local sim
2020-04-26 13:04:53 +03:00
tersec 8a72ae89b9
fix mainnet finalization (#906)
* fix mainnet finalization and swith eth2_network_simulation to a kind of small-mainnet profile

* Fix slot reference in trace logging

* bump a couple of spec refs from v0.11.0 to v0.11.1

* bump another spec ref to v0.11.1, one more try at Jenkins test vector download CI issue

* fix other slot reference in trace logging and skip past single-block/multi-slot gaps to re-approach from ancestry side by state_transitioning, by requiring exact match on both root hash and slot for fast path

* make more precise the fast path condition

* redo logic to make uniform with BeaconChainDB; fix chronos deprecation warning

* revert not-working replacement of deprecated chronos futures `or`

* switch testnet1 to mainnet
2020-04-20 19:27:52 +02:00