Commit Graph

219 Commits

Author SHA1 Message Date
tersec af3355e0f8
create local testnet mode for eth2_network (#1494) 2020-08-12 14:16:59 +00:00
tersec 22c1ef5a8d
split subscribe into non-validating subscribe and addValidator (#1485)
* split subscribe into non-validating subscribe and addValidator

* stop exporting get_committee_assignments
2020-08-11 15:08:44 +00:00
Jacek Sieka 280e72f3c9
remove snappy RPC support (#1477)
removed in 0.12.2 - the flow, in particular when the other peer doesn't
support snappy, is hard to follow because of the trial-and-error
approach - removing it simplifies things and removes some of the
hard-to-read parts of the thunking etc
2020-08-10 15:18:17 +02:00
Jacek Sieka 936440fccd
use libp2p peer events to track peer (#1468)
this resolves some peer counting issues that were happening because the
lifetime future in PeerInfo was unreliable (multiple PeerInfo instances
existed per peer)

In addition, this solves another race condition: when connecting to a
peer and later dialling that protocol, it is not certain that the same
connection will be used if there's a concurrent incoming peer connection
ongoing - better not make too many assumptions about who sent statuses
when.
2020-08-10 12:58:34 +02:00
Jacek Sieka 585c410d90 remove randompeers
unused, requires importing `random` which we're trying to avoid
2020-08-10 11:48:33 +03:00
Dmitriy Ryajov c5077af4bc
decreate amount of concurent dials (#1460) 2020-08-06 19:21:12 +00:00
Jacek Sieka f4c16ed0db
eh cleanups (#1458)
current exception sometimes buggy in nim
2020-08-06 18:47:39 +00:00
Jacek Sieka 221f372dbc use peer id in a number of places 2020-08-05 19:34:59 +03:00
Jacek Sieka d22a2cec2b
Start libp2p before writing ENR file (#1418)
this makes sure that all libp2p transports are open for business when
the file hits the ground
2020-08-03 19:35:27 +02:00
Dmitriy Ryajov 52d9d269d7
bump libp2p (delayed send, without hooks) (#1413)
* use `switch.isConnected`

* libp2p

* add timeout to publish

* use isConnected

* adjust timeouts

* latest libp2p master

* do not drop peers
2020-08-03 16:43:22 +00:00
tersec e0a6f58abe
convert 10 v0.12.1 spec refs to v0.12.2 (#1406) 2020-07-31 09:59:14 +00:00
Jacek Sieka c5fecd472f
more fork-choice fixes (#1388)
* more fork-choice fixes

* use target block/epoch to validate attestations
* make addLocalValidators sync
* add current and previous epoch to cache before doing state transition
* update head state using clearance state as a shortcut, when possible
* use blockslot for fork choice balances
* send attestations using epochref cache

* fix invalid finalized parent being used

also simplify epoch block traversal

* single error handling style in fork choice

* import fix, remove unused async
2020-07-30 17:48:25 +02:00
tersec 99f2d8e06c
update 14 v0.12.1 spec refs to v0.12.2 (#1400) 2020-07-30 09:47:57 +00:00
cheatfate 99dcb81e77 Initial commit. 2020-07-27 17:48:26 +03:00
tersec 90708a8287
Prefer converting int` to uint64 and switch foo.len.uint64 to .len64 (#1375)
* avoid converting from uint64 to int, and where most feasible, int type conversion at all

* .len.uint64 -> .len64

* fix 32-bit compilation

* try keeping state_sim loop variable/bounds as int for 32-bit Azure

* len64 -> lenu64
2020-07-26 20:55:48 +02:00
Ștefan Talpalaru c47532f2b0
deal with a temporary loss of network connectivity (#1354)
* don't kill the program if not connected to a bootstrap node within 30 seconds

* recover faster from loss of network connectivity

* connectWorker(): sleep 1s between dials

* launch_local_testnet.sh: increase BOOTSTRAP_TIMEOUT

* don't use metric value in program logic

* refactor some ungainly variable names
2020-07-23 22:51:56 +02:00
Eugene Kabanov 8c5aa7cbe7
Use only secp256k1 as identity in libp2p. (#1343)
* Add libp2p_pki_schemes to beacon_node and inspector configuration files.

* Fix inspector.nim.cfg file name.

* Do not allow beacon_node to be build without libp2p_pki_schemes option value.

* Fix compilation problems.

* Fix tests.

* Fix validator_client.
2020-07-21 18:07:14 +02:00
Dustin Brody 4140b3b9d9 update 29 spec refs to v0.12.1 2020-07-08 20:49:25 +00:00
Jacek Sieka f53425873c Only use noise 2020-07-08 08:05:38 +00:00
Jacek Sieka 6fe0a623f5
Crypto rng (#1284)
* use bearssl rng throughout

* bump

* bump

* move keygen out of crypto
2020-07-07 17:51:02 +02:00
Jacek Sieka ef2f037571
bump libp2p (#1267) 2020-07-01 13:41:40 +02:00
Jacek Sieka 816779733e
use the eth2 message id for gossip (in logging too) (#1246)
* use the eth2 message id for gossip (in logging too)

* bump

* add spec link
2020-06-28 22:06:50 +02:00
Jacek Sieka eeccaaf16d
stop gossipping non-snappy (#1240)
* stop gossipping non-snappy

Also simplify subscription and actually handle decoding errors

* log weird states too
2020-06-27 12:16:43 +02:00
Jacek Sieka 1301600341
Trusted blocks (#1227)
* cleanups

* fix ncli state root check flag
* add block dump to ncli_db
* limit ncli_db benchmark length
* tone down finalization logs

* introduce trusted blocks

We only store blocks whose signature we've verified in the database - as
such, there's no need to check it again, and most importantly, no need
to deserialize the signature when loading from database.

50x startup time improvement, 200x block load time improvement.

* fix rewinding when deposits have invalid signature
* speed up ancestor iteration by avoiding copy
* avoid deserializing signatures for trusted data
* load blocks lazily when rewinding (less memory used)

* chronicles workarounds

* document trustedbeaconblock
2020-06-25 12:23:10 +02:00
Jacek Sieka 1d709c09f4
secp: requiresInit (#1210)
* secp: requiresInit

* bump
2020-06-22 21:40:19 +02:00
Zahary Karadjov 14274587cf More user-friendly logging during mainchain monitoring 2020-06-22 17:30:04 +03:00
Eugene Kabanov 47eaaa7696
Fix connection workers race. (#1204) 2020-06-21 18:49:48 +02:00
Jacek Sieka a661ecbae1
bump libp2p (#1209) 2020-06-21 18:45:28 +02:00
Jacek Sieka 7e0e4dc327
don't crash on unknown disconnection reason, fix disconnection reason enum (#1208) 2020-06-20 09:24:33 +02:00
Jacek Sieka e813111b3b peers rpc call
simple way to display nbc peer table
2020-06-18 07:29:20 +00:00
Jacek Sieka 8fbbd59885
metric names (#1188)
* fix metric names to not clash with native libp2p metrics
* run testnet node with rpc enabled by default
2020-06-17 13:04:24 +02:00
kdeme a25bc025d1 Start discovery after starting libp2p switch 2020-06-16 13:33:46 +00:00
Kim De Mey 68a8b7d969
Filter discovery nodes on forkId (#1162) 2020-06-12 16:14:18 +02:00
Zahary Karadjov cf6a869e9e Address some TODO items; Handle start-up before genesis more properly 2020-06-11 17:40:08 +03:00
Zahary Karadjov c773e10c1a Attempt to reduce the risk of dropped network connections during the loading of KeyStores 2020-06-11 17:40:08 +03:00
Zahary Karadjov 25821331c4 More greppable code for the onPeerConnected operation 2020-06-11 17:40:08 +03:00
Jacek Sieka 016cc22173
show peer info on connect (#1155) 2020-06-11 07:14:26 +02:00
Eugene Kabanov 040e38529a
Fix #1140. (#1143)
SeenTable is now able to hold peers with different timeout values.
2020-06-08 18:02:50 +02:00
Eugene Kabanov 3ce98d5bca
Add checks for penalties which are not applied immediately. (#1139)
Change default maxHeadAge value to 1 epoch.
Set zero-point at the SyncQueue's initialization.
Remove annoying logs in runDiscoveryLoop.
2020-06-07 17:36:24 +02:00
Zahary Karadjov 0c78fc39e7
Use the latest LibP2P 2020-06-05 19:34:12 +03:00
Jacek Sieka bcbfa736c9
format ErrorMsg messages reasonably (#1109) 2020-06-04 08:19:25 +02:00
Jacek Sieka 56ffb696be
reorder ssz (#1099)
* reorder ssz

* split into hash_trees and ssz_serialization, roughly, for hashing and
IO
* move bitseqs into ssz (from stew)
* clean up imports

* docs, imports
2020-06-03 15:52:02 +02:00
cheatfate 405e9db199
Fix problem of good peers is also logged as timed out. 2020-06-03 13:48:01 +03:00
cheatfate 12e28a1fa9 Add proper concurrent connections.
Add SeenTable to avoid continuous attempts to dead peers.
Refactor onSecond.
Block backward sync while forward sync is working.
SyncManager now checks responses according corresponding requests + tests.
SyncManager now watching for not progressing local_head_slot and resets SyncQueue.
2020-06-03 12:53:57 +03:00
kdeme 06f025b228 Add timeout to switch.connect 2020-06-02 23:06:11 +03:00
Ștefan Talpalaru a90b0dd197
Merge pull request #1077 from status-im/timeout
Eth2Node.stop(): trace msg on timeout
2020-05-29 16:30:13 +02:00
Kim De Mey e33c8d9067
Bump nim-eth and accompanying discv5 cleanup (#1081) 2020-05-29 12:03:29 +02:00
Eugene Kabanov 21131e629b
Sync freeze fixes. (#1072)
* Add ability to reset state of sync manager.
Fix bug when sync got stuck on `zero-point` reset.
Fix bug when sync got stuck when some of the workers waiting for failing one.

* Remove debugging comments and imports.

* Remove not used pendingLock.
2020-05-28 07:02:28 +02:00
Ștefan Talpalaru 273a912ae0
Eth2Node.stop(): trace msg on timeout 2020-05-28 03:14:01 +02:00
Zahary Karadjov 28128f4d2f Add a handler for the Goodbye message
The lack of body of `goodbye` in sync_protocol.nim was preventing
the respective LibP2P protocol to be mounted and advertised on the
network.

Adding a body fixes that, but I've also made some changes in the
P2P protocol codegen that will prevent the issue from happening
again (no body is now considered the equivalent of having an empty
body).
2020-05-26 22:17:26 +03:00
Zahary Karadjov 833f19e942 Reform the networking layer in order to handle the new stricter SSZ API 2020-05-24 19:00:34 +03:00
Zahary Karadjov accd5fe954 Don't use StackArray in ssz; Drop the support for strings 2020-05-24 19:00:34 +03:00
Ștefan Talpalaru b2193f1b8f
Eth2Node.stop(): 5s timeout 2020-05-21 00:06:01 +02:00
Jacek Sieka a38eddcaac
remove ssz stint support (#1046) 2020-05-20 19:05:22 +02:00
Ștefan Talpalaru 383b22795c
bump submodules (#1043) 2020-05-20 06:57:39 +02:00
Ștefan Talpalaru c4462af4ab
beacon_node: graceful shutdown (#1033)
* beacon_node: graceful shutdown

* separate BeaconNodeStatus and BeaconNode instances
2020-05-19 20:57:35 +02:00
Dmitriy Ryajov 0649d47df0 use proper transport flags 2020-05-18 21:51:03 +00:00
Jacek Sieka 6be7d64e8c
bump libp2p (#1031) 2020-05-18 10:11:21 +02:00
Zahary Karadjov 24a17f5814 Fix an RPC error in Lighthouse triggered by the getMetadata message 2020-05-16 09:56:13 +03:00
tersec 74db0f3c8d
fix some XDeclaredButNotUsed hints (#1027) 2020-05-15 14:41:00 +02:00
Zahary Karadjov 75c1c6a95c Enable Snappy by default (using LibP2P steams for now)
This refactors the newly added Snappy streaming back-ends trying to
make them more similar and to reduce the code duplication to a minimum.
2020-05-13 12:18:42 +03:00
Zahary Karadjov f055fad08a Make the Snappy FastStreams integration optional by duplicating it for LibP2P streams 2020-05-13 12:18:42 +03:00
Zahary Karadjov 15f0153441 Cosmetic improvements 2020-05-13 12:18:42 +03:00
Zahary Karadjov 9538b60704 Integrate the async Snappy implementation 2020-05-13 12:18:42 +03:00
Zahary Karadjov a739d7e8d6 Adapt SSZ to the latest FastStreams API 2020-05-13 12:18:42 +03:00
Eugene Kabanov da0b1a4993
Fix status handling. (#1008)
* Fix status handling.
Add log map of received blocks.

* Fix review comments.
Fix UnusedImport in sync_protocol.nim
2020-05-13 08:37:58 +02:00
Jacek Sieka 592d6f65c3
bump (#1001) 2020-05-09 16:47:14 +02:00
Ștefan Talpalaru fc9d6bc1b3
enable SO_REUSEADDR in libp2p transports 2020-05-08 21:50:20 +02:00
Eugene Kabanov 3ffd0d7216
Fix gossip messages not delivered. (#983) 2020-05-06 15:42:59 +00:00
Zahary Karadjov 7237a370ba Implement https://github.com/status-im/nim-beacon-chain/issues/872
Please see the newly added 'schlesi-dev' Makefile target.
It demonstrates how the log level can be specified for individual topics.

Additionally, when connecting to testnets like 'schlesi' there will be
two additional log files produced in the working directory:

* json-log.txt
* text-log.txt (in the textblocks format)
2020-04-28 13:49:30 +03:00
Viktor Kirilov dcc70c5259 resolved final comment from review 2020-04-27 22:10:35 +03:00
Viktor Kirilov 8ef28c905d fixed commets from review 2020-04-27 22:10:35 +03:00
Viktor Kirilov 5ecba6df49 broadcasting and listening to the '_snappy' topics as well 2020-04-27 22:10:35 +03:00
Zahary Karadjov 740b76d152 Spec-compliant implementation of Eth1 monitoring; Eth1-enabled local sim
BEWARE! This commit will trigger a stack overflow during local sim
2020-04-26 13:04:53 +03:00
Eugene Kabanov be89a3c54d
Add "drop by score" ability to PeerPool. (#917)
* Add "drop by score" ability to PeerPool.
Add tests.
Fix syncmanager queue to start from most fresh data.

* Fix endless cycle at the end of syncing process.
2020-04-23 17:31:00 +02:00
Eugene Kabanov 3d42da90a8
Syncing. (#909) 2020-04-20 16:59:18 +02:00
Jacek Sieka 8b6c51e78d
secp: convenience api (#897) 2020-04-17 15:29:49 +02:00
Zahary Karadjov a4830976ab
hotfix: fix the selecting of the syncing protocol handlers 2020-04-16 15:01:19 +03:00
Zahary Karadjov 672f690bf6 Implement most of the v0.11.1 networking changes 2020-04-16 01:01:18 +03:00
Zahary Karadjov 22e5f8165e Don't use `defer` in eth2_network to avoid https://github.com/nim-lang/Nim/issues/13899 2020-04-16 01:01:18 +03:00
Zahary Karadjov f34fd46e3a Add snappy and protobuf-serialization; Use the latest FastStreams API 2020-04-16 01:01:18 +03:00
tersec 31bf8fa408
remove all but one UnusedImport warning (#881)
* remove all but one UnusedImport warning

* bump a few more spec version references from v0.10.1 to v0.11.1

* more v0.10.1 spec reference updates/removals

* yet more v0.10.1 spec reference updates
2020-04-10 16:06:24 +02:00
Jacek Sieka 04cf6d30eb
vendor: bump nim-eth + fallout (#861) 2020-04-05 11:50:31 +02:00
Jacek Sieka d180724d41
Bump eth (#857)
* vendor: bump stew
* vendor: bump nim-eth
* vendor: bump web3 too
2020-04-03 11:12:38 +02:00
kdeme fc2fff249b Discv5: Don't pass ip address when external ip is not known 2020-04-02 00:52:34 +03:00
tersec cd388bc9bb
extended validation (#812)
* initial extended validation setup

* flesh out all TODO items for attestation and beaconblock verification

* fix finalization and add chronicles debugging messages

* directly use blockPool.headState rather than pointlessly updating it and document this constraint

* fix logic relating to first-attestation checking; support validating blocks across multiple forks
2020-03-31 18:39:02 +00:00
Ștefan Talpalaru 97abe77398
close unused PeerInfo instance
Should not be needed from the GC's point of view, but needed when
gathering Future.state statistics, due to `PeerInfo.lifefut`.
2020-03-30 00:23:01 +02:00
Zahary Karadjov 46e395c647 Switch to GossipSub 2020-03-25 23:32:05 +02:00
Zahary Karadjov 86a3bea1ec Use the latest Chronos and Libp2p 2020-03-24 22:14:40 +02:00
kdeme b80a5b90df Adjustments to store bootnode enrs in discovery object + bump nim-eth 2020-03-24 22:14:40 +02:00
Zahary Karadjov f2434139e9
Hotfix: Don't report failure to connect to bootstrap node due to the initial sync taking more than 30 seconds 2020-03-23 22:29:50 +02:00
cheatfate d63e4870e6 Fix Stream EOF problem. 2020-03-23 22:21:27 +02:00
Zahary Karadjov 46a3009c63 Restore the address file feature (and use it for synchronization in local sim) 2020-03-23 14:59:33 +02:00
Zahary Karadjov 4623aa81ec Clean up some obsolete code 2020-03-23 14:59:33 +02:00
Zahary Karadjov d5e4e640b4 Merge libp2p_backend into eth2_network 2020-03-23 14:59:33 +02:00
Zahary Karadjov 0c018cb68a Mechanically remove all mentions of the daemon from the code 2020-03-23 14:59:33 +02:00
Zahary Karadjov d2b7ee27de The RPC service listens on localhost by default 2020-03-22 19:00:09 +02:00
Zahary Karadjov 19847bbd20 Remove some warnings 2020-03-22 19:00:09 +02:00
cheatfate d802fa593c Add pubsub message logging.
Initial commit of logtrace tool.
2020-03-11 17:26:30 +00:00
cheatfate ab1a08084b One more attempt to fix compilation error. 2020-02-28 14:44:41 +02:00