Commit Graph

195 Commits

Author SHA1 Message Date
Zahary Karadjov ba59dd85cd Support for fallback web3 providers; Fix resource leaks on Eth1 monitor restarts 2021-04-08 12:46:42 +03:00
Zahary Karadjov 7451c4171f
v1.1.0-rc; Enable v2 doppelganger detection by default 2021-04-07 00:38:47 +03:00
Jacek Sieka beceb060c4
Write state diffs to separate table (and experimentally, files instead of db) (#2460) 2021-04-06 21:56:45 +03:00
Jacek Sieka 8f441588cd
Add a few help defaults, format all of them the same (#2284) (#2467)
No support in confutils, so so this code simply hardcodes it
2021-04-01 12:28:57 +00:00
Jacek Sieka 2695cfa864
EH cleanup (#2455)
almost 100% raises in nimbus-eth2 now!

* fix some rare exception-related crashes in json-rpc
2021-03-26 07:52:01 +01:00
Giovanni Petrantoni 9558946df4
Expose gossip direct peers (#2246)
* Expose gossip direct peers

* direct-peer fixes

* add explicit info about direct peer

* build fixes wip
2021-03-23 15:10:17 +09:00
Mentor Palokaj 895fe4baf1
Make it clear that the default port for ETH2 is 9000 udp/tcp (#2421)
* Add default port info

* Add default port info

* Edit CLI and docs to reflect 9000 port

* Apply suggestions from code review

Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>
2021-03-22 08:22:56 +01:00
Zahary Karadjov 77ee210762
v1.0.10 2021-03-10 23:33:37 +02:00
Zahary Karadjov 196e3ea6b6
Switch to slashing DB v2 by default 2021-03-09 00:47:29 +02:00
kdeme 5f750f84b4 Use setupAddress for better IP and ports configuration 2021-03-06 00:09:09 +02:00
Mamy Ratsimbazafy d47f53cd9d
Reorg (5/5) (#2377)
* Reorg things left into networking and gossip_processing

* time -> beacon_clock

* fix builds
2021-03-05 14:12:00 +01:00
Jacek Sieka 0dbc7162ac startup cleanup
* fix several memory leaks due to temporaries not being reset during
init
* avoid massive main() function with lots of stuff in it
* disable nim-prompt (unused)
* reuse validator pool instance in eth2_processor
* style cleanup
2021-02-22 23:32:54 +02:00
Zahary Karadjov e1d6df1e5d Continue using the V1 Slashing DB by default 2021-02-20 22:46:35 +02:00
Mamy Ratsimbazafy 5daafd480f
Slashing protection updates (#2333)
* Fix slashing protection always try to migrate at startup
* Add CLI option for dual DB
2021-02-19 17:18:17 +02:00
Dustin Brody 67e4a045a3 simplify doppelganger detection to boolean 2021-02-03 20:55:33 +02:00
Kim De Mey 73d9c2aa53
Add enr auto update cli option and bump nim-eth (#2278) 2021-02-02 09:07:21 +01:00
tersec 6141286547
rename doppelganger protection to doppelganger detection; switch default from warn to stop (#2281) 2021-02-01 12:18:16 +01:00
Zahary Karadjov fa99c3b417 Fix #2261
Also bumps Confutils to allow setting the hidden --web3-mode param
(to allow testing the eth1 syncing without validators)
2021-01-30 01:32:20 +02:00
Dustin Brody 281853eee8 rename options and internal structures to doppelgangerFoo and remove probing 2021-01-30 00:17:54 +02:00
Dustin Brody a16f5afcd5 pre-emptive duplicate validator detection heuristic 2021-01-21 16:03:02 +02:00
Zahary Karadjov c8c819359c
More clear error message when a validator exit was rejected 2021-01-15 19:40:05 +02:00
tersec dde973e2d4
allow always-on subscription to all attestation subnets when gossiping (#2225)
* allow always-on subscription to all attestation subnets when gossiping

* in subscribe-all-subnets mode, consider all subnets to be stability subnets for ENR purposes
2021-01-12 13:43:15 +01:00
Zahary Karadjov 983b3c9fbf Add a 'we3 test' command for verifying the compatibility of a web3 provider 2020-12-10 02:54:58 +02:00
Zahary Karadjov 3c0dfc2fbe Implement the 'deposits exit' command; Remove 'deposits create' 2020-11-29 23:08:07 +02:00
Zahary Karadjov ae19ab72c0 Implement #2067 2020-11-29 18:27:26 +02:00
Jacek Sieka c3bfe15a35
increase default max peers (#2106)
* increase default max peers

also avoid reconnection when opening stream as this might induce
a loop

* Use dial without addresses

* dial back max peers a little
2020-11-29 14:43:41 +01:00
Zahary Karadjov bf2673abc4
Restore the ETH display in the status bar 2020-11-28 20:53:51 +02:00
Eugene Kabanov db030eb944
Setup agent/proto version identifiers in libp2p. (#2079)
* Supply `nim-libp2p` with `agent-string` and `proto-string` as configured parameters.

* Remove protoString and rng check.

* Fix compilation problem.
2020-11-28 08:00:36 +01:00
kdeme e69b5ff473 Add a record create and print command 2020-11-25 18:32:59 +02:00
Zahary Karadjov 80ca7a2d9f Implement #1768 2020-11-12 22:29:43 +02:00
Jacek Sieka 332e248d91
rpc: setLogLevel (#1991)
* rpc: setLogLevel

also change default rpc port to match makefile and manual

* set topic-level formats too

and make feature less secret
2020-11-12 11:46:02 +01:00
Zahary Karadjov 4b603d7f5a Change the default log level to INFO 2020-11-09 11:38:52 +02:00
Zahary Karadjov e9b9cd75ee Rename binaries; Mimic the original repo layout in the distribution 2020-11-09 11:38:52 +02:00
Eugene Kabanov eee01a32d5
Regression fix of eth2_network_simulation on Windows. (#1900)
* Concentrate all sensitive writeFile/createPath calls in one place.
Fix eth2_network_simulation for Windows.

* Remove artifacts.

* fix import

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-10-27 12:04:17 +01:00
Zahary Karadjov 02a3770803 Adds a fuzzing test for the validator client CLI
Also removes an obsolete way to add validators through the command line
2020-10-15 20:15:51 +03:00
Zahary Karadjov deaddc1fc0 Address review comments 2020-10-07 09:32:03 +03:00
Zahary Karadjov aed291128a Add support for starting from weak subjectivity checkpoints
Also removes the `genesis.ssz` file stored in the data folder.
The `medalla-fast-sync` target has been adapted to use the new features.
2020-10-07 09:32:03 +03:00
Zahary Karadjov fec4b5014d Address #1694 2020-10-06 15:46:22 +03:00
cheatfate 82228fe471 Fix last createDir. 2020-10-05 22:19:50 +03:00
cheatfate add22a20e1 Update local_testnet and simulation scripts to use netkey-file and insecure-netkey-password.
Add more logging
2020-10-05 22:19:50 +03:00
cheatfate e1182f8000 Add insecure password for automated testing.
Fix checkDataDir to run before setupLogging.
2020-10-05 22:19:50 +03:00
cheatfate e43af4e978 Fix CLI option. 2020-10-05 22:19:50 +03:00
cheatfate c5c788a9db Secure network key file and data directory. 2020-10-05 22:19:50 +03:00
Zahary Karadjov f5340998b9 Add a fuzzing test for the beacon node CLI parser 2020-10-02 12:42:32 +03:00
Kim De Mey 23bec99341
Let also discovery used listen-address cli option instead of always any address (#1658) 2020-09-27 22:00:24 +02:00
Viktor Kirilov e926db22b9 initial implementation of attestation aggregation for the validator API 2020-09-24 11:31:14 +03:00
Eugene Kabanov c7c9b9d5f1
Syncing V2 (#1602)
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* Remove suspects field.
2020-09-11 14:46:01 +02:00
Viktor Kirilov d9f9949ef0 use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process 2020-09-02 12:47:00 +03:00
Viktor Kirilov 65d7787b1e 50/50 bn/vc split for the validator keys ON by default for the testnet scripts 2020-09-01 16:39:07 +03:00
Jacek Sieka a7a279d615
add option to disable discv5 (#1509) 2020-08-24 13:52:06 +02:00
Jacek Sieka 10da7fe9da remove eth from default status bar
not viable at higher validator counts - linear scan + forced public key
init makes it extremely slow
2020-08-10 17:01:53 +03:00
Zahary Karadjov c293254ded Add 'deposits import' command; Switch to NJS when loading the keystores and improve the data validation 2020-08-02 23:00:43 +03:00
Zahary Karadjov d58668157a Use consistent naming for the synonyms 'deposits-dir' and 'validators-dir' 2020-08-02 23:00:43 +03:00
Zahary Karadjov 0c60e452e7 Allow the deposit contract deployment block to be specified as a number (needed for Medalla) 2020-07-28 18:13:01 +03:00
Zahary Karadjov 40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
Zahary Karadjov fcd412f7a1 Finish the 'create wallet' command; Addresses #1319 2020-07-28 07:36:25 +03:00
Ștefan Talpalaru e9193fc9da
eth2_network_simulation: fixes (#1368)
- delete "tests/simulation/{data,validators}" by default, because old
  validator keys can lead to a crash with a cryptic error message
- actually start the Prometheus daemon on `make eth2_network_simulation`
- kill any running Prometheus daemon on exit
- fix some shell script syntax incompatible with Bash
2020-07-23 17:58:54 +02:00
zah 8970a22fdd
Add a non-optional Json file log for the beacon node as stopgap measure (#1322)
* Add a non-optional Json file log for the beacon node as stopgap measure

* Fix the build with -d:testnet_servers_image
2020-07-15 15:15:55 +02:00
Zahary Karadjov 93b04bc214 Add an option for graffiti customization 2020-07-12 21:01:31 +03:00
Zahary Karadjov 87928e0069
Documentation; 'deposits create' can now ask for a private key 2020-07-10 18:18:14 +03:00
Zahary Karadjov 540b2828b2
Adapt the local sim scripts to use the new run-time presets 2020-07-10 01:08:54 +03:00
Zahary Karadjov c4af4e2f35
Working test suite with run-time presets 2020-07-08 02:02:14 +03:00
Zahary Karadjov 98761afc64
Run beacon_node --network=altona without initial Eth1 syncing 2020-07-02 18:52:48 +03:00
Zahary Karadjov e342b96d2e
Embed the Altona metadata in the NBC executable 2020-07-02 18:14:11 +03:00
Zahary Karadjov 62e4efe34b Re-wire the sending of deposits by the 'deposits create' command 2020-06-24 23:06:05 +03:00
Zahary Karadjov 384e512031 Implement EIP-2386 wallets; Progress towards a CLI for interactive wallet creation
For more information:
4494da0966/EIPS/eip-2386.md (specification)
2020-06-24 23:06:05 +03:00
Ștefan Talpalaru bdd56b626f
Merge pull request #1206 from status-im/readme
Update readme and manual
2020-06-24 01:04:25 +02:00
Zahary Karadjov 7211a99086 Smooth out the differences between Ganache and Infura; Working local sim and Altona target 2020-06-22 17:30:04 +03:00
Zahary Karadjov 1def383ad1 Reword the Tmux setup to handle all simulation scenarios in more visible way 2020-06-22 17:30:04 +03:00
Zahary Karadjov e9d68e2f7b Add simulation mode that bootstraps from an Eth1 ganache instance 2020-06-22 17:30:04 +03:00
李婷婷 fd35b61ecb fix typo 2020-06-19 23:00:18 +08:00
Jacek Sieka 49e9167b28 clean up dump feature
* don't write blocks that get added to database
* don't write states
* write to folders
* add state dumping feature to `ncli_db` to get any known state from the
database
2020-06-16 13:44:37 +00:00
Zahary Karadjov 6e88a07b51 Fix the validator client build 2020-06-11 17:40:08 +03:00
Zahary Karadjov 2acda1c115 Provide a default value for secretsDir (similar to validatorsDir) 2020-06-11 17:40:08 +03:00
Zahary Karadjov a8113cf2bc Restore the local sim to a working state 2020-06-11 17:40:08 +03:00
Zahary Karadjov 17343442ea Implement more of the KeyStore spec and integrate it in the beacon node 2020-06-11 17:40:08 +03:00
Viktor Kirilov 3bae40ae91 - extracted the commands to run a VC into a separate run_validator.sh script
- each BN gets half of its previous validators as inProcess and the other half goes to the respective VC for that BN - using separate data dirs where the keys are copied
    - also removed a few command line options which are no longer necessary
- block proposals originating from a VC are propagated from one BN to the rest properly
- other cleanup & moving code back to  since it is no longer used elsewhere
2020-06-10 13:50:50 +03:00
Jacek Sieka 1e71388093
Include finalization, head and time (in that order) in standard bar (#1142)
These are the 3 most important things in the life of a beacon-node -
finalization checkpoint for security, head + time for seeing that we're
synced and working.

The order given is the expected increasing order that things should be
moving in - finalization lags head, and head lags clock - the structure
helps the user remember, but also see the pattern where time is getting
out of sync with head.

A validator would also want to know when the next action is going to be
(ie next attestation / block production) but that's for another day.
2020-06-08 18:04:11 +02:00
Jacek Sieka bb698a7e16
lots of peers (#1130) 2020-06-05 22:25:01 +02:00
Zahary Karadjov 0c78fc39e7
Use the latest LibP2P 2020-06-05 19:34:12 +03:00
Jacek Sieka f53b55cbe0 SSZ cleanup
* be stricter about SSZ length prefix
* compute zeroHash list at compile time
* remove SSZ schema stuff
* move SSZ navigation to ncli
* cleanup a few leftover openArray uses
2020-05-28 19:15:16 +03:00
Viktor Kirilov cefd525ab3 more work on the BN/VC split
- fixed comments from the last review
- getting more data VIA RPC, moved some code back into the BN only
- attestation duties being requested as well
2020-05-28 13:01:42 +03:00
Viktor Kirilov 8760494c72 first batch of work towards the VC/BN split:
- we have a new binary which connects via RPC to the respective BN and has an internal clock - waking it up on every slot
- the BN has a new option called --external-validators and currently in order to have the VC binaries to run we need to pass EXTERNAL_VALIDATORS=yes to make
- factored some code out of beacon_node.nim for easier reuse in validator_api.nim and validator_client.nim
- the VC loads its associated private keys from the datadir for its BN
- most of the validator API calls have been implemented as a stub.
- the VC polls its BN at the start of each epoch - getting a list of all active validators for the current epoch - and then continues to request blocks and sign them with its appropriate validators when necessary
2020-05-25 16:23:15 +03:00
Zahary Karadjov 7237a370ba Implement https://github.com/status-im/nim-beacon-chain/issues/872
Please see the newly added 'schlesi-dev' Makefile target.
It demonstrates how the log level can be specified for individual topics.

Additionally, when connecting to testnets like 'schlesi' there will be
two additional log files produced in the working directory:

* json-log.txt
* text-log.txt (in the textblocks format)
2020-04-28 13:49:30 +03:00
Zahary Karadjov 740b76d152 Spec-compliant implementation of Eth1 monitoring; Eth1-enabled local sim
BEWARE! This commit will trigger a stack overflow during local sim
2020-04-26 13:04:53 +03:00
Jacek Sieka ed74770451
spec: regulate exceptions (#913)
* spec: regulate exceptions

* a few more simple raises
2020-04-22 07:53:02 +02:00
Zahary Karadjov d2b7ee27de The RPC service listens on localhost by default 2020-03-22 19:00:09 +02:00
Zahary Karadjov adcec61081 Initial implementation of a JSON-RPC service 2020-03-22 19:00:09 +02:00
Mamy André-Ratsimbazafy bfd4df912e Update to BLS v0.10.1 2020-03-13 12:20:41 +00:00
Zahary Karadjov 7f395e554a WIP Working local network sim with discovery
* The bootstrap_nodes.txt file in the node's data dir is now optional
* Log more data on start-up
* Use the latest ENR APIs
* Fix simulation build errors
2020-02-18 12:53:49 +02:00
Zahary Karadjov ee7c2c9dff Unify the bootstrap nodes handling code
We no longer discriminate between ENR, MultiAddress or ENode
bootstrap records (all of them are remapped to ENodes).

The discovery loop will stochastically try to reconnect to
accidentally disconnected nodes.
2020-02-18 12:53:49 +02:00
Dustin Brody bde1eefdcf improve option naming 2020-02-17 22:00:41 +00:00
Dustin Brody e13846bec6 allow beacon_node to verify finalization when appropriate, and enable by default for make eth2_network_simulation 2020-02-17 22:00:41 +00:00
Zahary Karadjov 07d329acb8 Wire-up the peer dialing loop and introduce --max-peers option 2020-01-21 01:26:57 +02:00
Zahary Karadjov 52878405b7 Integrate Discovery V5 and support ENR bootstrap records 2020-01-21 01:26:57 +02:00
Jacek Sieka a3145f0f5c cleanups
* move stuff out of beaconnode
* fix use-after-free in query
2019-12-10 14:47:52 +00:00
Jacek Sieka afc0686b36 save ssz dummps during processing (#471) 2019-12-03 11:32:27 +00:00
Zahary Karadjov 4c2ded25a6 Add detailed tracing for blocks by range requests 2019-11-26 02:45:25 +02:00
Zahary Karadjov ac5aabd569 Use the latest confutils 2019-11-11 23:29:36 +00:00
Zahary Karadjov ed9cb274a2 Restore a change mysteriously lost during rebasing 2019-11-11 23:29:36 +00:00