Commit Graph

191 Commits

Author SHA1 Message Date
tersec a2270a5f27
update to v0.12.3 message ID for Spadina launch (#1762)
* update to v0.12.3 message ID for Spadina launch

* remove base64 import
2020-09-28 17:07:10 +02:00
Kim De Mey 23bec99341
Let also discovery used listen-address cli option instead of always any address (#1658) 2020-09-27 22:00:24 +02:00
Jacek Sieka 7837646079
anonymize libp2p messages (#1756)
* anonymize libp2p messages

* bump
2020-09-25 18:40:30 +02:00
Eugene Kabanov 1bf8d3af33
Disconnect peers with low score. (#1747)
* Disconnect peers with low score.

* Change PeerScoreLow value.

* Add spec url for DisconnectionReason.
2020-09-25 15:43:45 +02:00
tersec f96ad87d28
switch another 50+ spec refs from v0.12.2 to v0.12.3 (#1749) 2020-09-25 11:52:50 +00:00
Jacek Sieka b3a9afa0b1
libp2p: limit max gossip writes (#1739)
* libp2p: limit max gossip writes

* bump
2020-09-24 19:03:17 +02:00
Jacek Sieka e1c177cdd1
bump libp2p (#1721)
gossipsub 1.1 can be enabled with -d:nbc_gossipsub_11
2020-09-22 19:34:34 +02:00
Eugene Kabanov 654b8d66bf
Peer management (#1707)
* addPeer() and addPeerNoWait() now returns PeerStatus, not bool.
Minor refactoring of PeerPool.
Fix tests.

* Refactor PeerPool.
Add lenSpace.
Add tests for lenSpace.
PeerPool.add procedures now return different error codes.
Fix SyncManager break/continue problem.
Fix connectWorker break/continue problem.
Refactor connectWorker and discoveryLoop.
Fix incoming/outgoing blocking problem.

* Refactor discovery loop.
Add checkPeer.

* Fix logic and compilation bugs.

* Adjust position of debugging log.

* Fix issue with maximum peers in PeerPool.
Optimize node record decoding.

* fix discoveryLoop.

* Remove aliases and fix tests using aliases.
2020-09-21 18:02:27 +02:00
Jacek Sieka fc10f5121a protect against data after initial request
spec requires that channel is closed

also, avoid some unnecessary futures
2020-09-18 21:34:07 +03:00
tersec e106549efe
keep REJECT/IGNORE of messages failing validation for libp2p scoring (#1676)
* keep REJECT/IGNORE status of messages failing validation for libp2p scoring

* fix test suite
2020-09-18 13:53:09 +02:00
Dmitriy Ryajov 2f89e2ab4e
drop subscribePeer, it's called from pubsub now (#1677) 2020-09-17 11:40:21 +02:00
Eugene Kabanov 6e463257f4
PeerPool fixes. (#1654)
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.

* Remove unused imports.

* Add test for a bug.

* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
2020-09-16 13:00:11 +03:00
Jacek Sieka c76305f824
fix some todo (#1645)
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
2020-09-14 14:50:03 +00:00
Jacek Sieka a087909683
fix req/resp protocol (#1621)
per spec, we must half-close request stream - not doing so may lead to
failure of the other end to start processing our request leading to
timeouts.

In particular, this fixes many sync problems that have been seen on
medalla.

* remove safeClose - close no longer raises
* use per-chunk timeouts in request processing
2020-09-10 21:40:09 +02:00
tersec d0de1a49a3
Fix some warnings and hints and partly revert #1610 (#1615)
* address some XDeclaredButNotUsed, ConvFromXtoItselfNotNeeded, and UnusedImport hints and warnings

* partly revert #1610
2020-09-08 11:32:43 +00:00
Jacek Sieka d584591ded
simplify libp2p logging (#1605)
and a few other small logging fixes
2020-09-06 10:39:25 +02:00
Jacek Sieka a7a279d615
add option to disable discv5 (#1509) 2020-08-24 13:52:06 +02:00
Jacek Sieka 22998fdfd4 avoid double deserialization
When blocks and attestations arrive, they are SSZ-decoded twice: once
for validation and once for processing. This branch enqueues the decoded
block directly for processing, avoiding the second, slow
deserialization.

* move processing of blocks and attestations to queue
* ...and out from beacon_node
* split attestation processing into attestations and aggregates
  * also updates metrics
* clean up logging to better follow the lifetime of gossip: arrival,
validation and processing
* drop attestations and aggregates if there are too many
* try to prioritise blocks and aggregates before single-validator
attestations
2020-08-21 11:46:25 +03:00
Zahary Karadjov 2c19e3f8cd
[skip ci] Use GOSSIP_MAX_SIZE when snappy decoding in the inspector as well; Bumps 2020-08-19 14:33:52 +03:00
Zahary Karadjov 3433c77c35 Prevent Snappy decompression bombs 2020-08-19 10:13:04 +03:00
Dmitriy Ryajov 87f983c639 use split out pubsub 2020-08-17 17:24:36 +03:00
tersec 612881b95d
refactor topic (un)subscribing/validating to collate each (#1510)
* refactor topic (un)subscribing/validating to collate each

* fix comment

* tweak comment
2020-08-17 14:07:29 +02:00
tersec af3355e0f8
create local testnet mode for eth2_network (#1494) 2020-08-12 14:16:59 +00:00
tersec 22c1ef5a8d
split subscribe into non-validating subscribe and addValidator (#1485)
* split subscribe into non-validating subscribe and addValidator

* stop exporting get_committee_assignments
2020-08-11 15:08:44 +00:00
Jacek Sieka 280e72f3c9
remove snappy RPC support (#1477)
removed in 0.12.2 - the flow, in particular when the other peer doesn't
support snappy, is hard to follow because of the trial-and-error
approach - removing it simplifies things and removes some of the
hard-to-read parts of the thunking etc
2020-08-10 15:18:17 +02:00
Jacek Sieka 936440fccd
use libp2p peer events to track peer (#1468)
this resolves some peer counting issues that were happening because the
lifetime future in PeerInfo was unreliable (multiple PeerInfo instances
existed per peer)

In addition, this solves another race condition: when connecting to a
peer and later dialling that protocol, it is not certain that the same
connection will be used if there's a concurrent incoming peer connection
ongoing - better not make too many assumptions about who sent statuses
when.
2020-08-10 12:58:34 +02:00
Jacek Sieka 585c410d90 remove randompeers
unused, requires importing `random` which we're trying to avoid
2020-08-10 11:48:33 +03:00
Dmitriy Ryajov c5077af4bc
decreate amount of concurent dials (#1460) 2020-08-06 19:21:12 +00:00
Jacek Sieka f4c16ed0db
eh cleanups (#1458)
current exception sometimes buggy in nim
2020-08-06 18:47:39 +00:00
Jacek Sieka 221f372dbc use peer id in a number of places 2020-08-05 19:34:59 +03:00
Jacek Sieka d22a2cec2b
Start libp2p before writing ENR file (#1418)
this makes sure that all libp2p transports are open for business when
the file hits the ground
2020-08-03 19:35:27 +02:00
Dmitriy Ryajov 52d9d269d7
bump libp2p (delayed send, without hooks) (#1413)
* use `switch.isConnected`

* libp2p

* add timeout to publish

* use isConnected

* adjust timeouts

* latest libp2p master

* do not drop peers
2020-08-03 16:43:22 +00:00
tersec e0a6f58abe
convert 10 v0.12.1 spec refs to v0.12.2 (#1406) 2020-07-31 09:59:14 +00:00
Jacek Sieka c5fecd472f
more fork-choice fixes (#1388)
* more fork-choice fixes

* use target block/epoch to validate attestations
* make addLocalValidators sync
* add current and previous epoch to cache before doing state transition
* update head state using clearance state as a shortcut, when possible
* use blockslot for fork choice balances
* send attestations using epochref cache

* fix invalid finalized parent being used

also simplify epoch block traversal

* single error handling style in fork choice

* import fix, remove unused async
2020-07-30 17:48:25 +02:00
tersec 99f2d8e06c
update 14 v0.12.1 spec refs to v0.12.2 (#1400) 2020-07-30 09:47:57 +00:00
cheatfate 99dcb81e77 Initial commit. 2020-07-27 17:48:26 +03:00
tersec 90708a8287
Prefer converting int` to uint64 and switch foo.len.uint64 to .len64 (#1375)
* avoid converting from uint64 to int, and where most feasible, int type conversion at all

* .len.uint64 -> .len64

* fix 32-bit compilation

* try keeping state_sim loop variable/bounds as int for 32-bit Azure

* len64 -> lenu64
2020-07-26 20:55:48 +02:00
Ștefan Talpalaru c47532f2b0
deal with a temporary loss of network connectivity (#1354)
* don't kill the program if not connected to a bootstrap node within 30 seconds

* recover faster from loss of network connectivity

* connectWorker(): sleep 1s between dials

* launch_local_testnet.sh: increase BOOTSTRAP_TIMEOUT

* don't use metric value in program logic

* refactor some ungainly variable names
2020-07-23 22:51:56 +02:00
Eugene Kabanov 8c5aa7cbe7
Use only secp256k1 as identity in libp2p. (#1343)
* Add libp2p_pki_schemes to beacon_node and inspector configuration files.

* Fix inspector.nim.cfg file name.

* Do not allow beacon_node to be build without libp2p_pki_schemes option value.

* Fix compilation problems.

* Fix tests.

* Fix validator_client.
2020-07-21 18:07:14 +02:00
Dustin Brody 4140b3b9d9 update 29 spec refs to v0.12.1 2020-07-08 20:49:25 +00:00
Jacek Sieka f53425873c Only use noise 2020-07-08 08:05:38 +00:00
Jacek Sieka 6fe0a623f5
Crypto rng (#1284)
* use bearssl rng throughout

* bump

* bump

* move keygen out of crypto
2020-07-07 17:51:02 +02:00
Jacek Sieka ef2f037571
bump libp2p (#1267) 2020-07-01 13:41:40 +02:00
Jacek Sieka 816779733e
use the eth2 message id for gossip (in logging too) (#1246)
* use the eth2 message id for gossip (in logging too)

* bump

* add spec link
2020-06-28 22:06:50 +02:00
Jacek Sieka eeccaaf16d
stop gossipping non-snappy (#1240)
* stop gossipping non-snappy

Also simplify subscription and actually handle decoding errors

* log weird states too
2020-06-27 12:16:43 +02:00
Jacek Sieka 1301600341
Trusted blocks (#1227)
* cleanups

* fix ncli state root check flag
* add block dump to ncli_db
* limit ncli_db benchmark length
* tone down finalization logs

* introduce trusted blocks

We only store blocks whose signature we've verified in the database - as
such, there's no need to check it again, and most importantly, no need
to deserialize the signature when loading from database.

50x startup time improvement, 200x block load time improvement.

* fix rewinding when deposits have invalid signature
* speed up ancestor iteration by avoiding copy
* avoid deserializing signatures for trusted data
* load blocks lazily when rewinding (less memory used)

* chronicles workarounds

* document trustedbeaconblock
2020-06-25 12:23:10 +02:00
Jacek Sieka 1d709c09f4
secp: requiresInit (#1210)
* secp: requiresInit

* bump
2020-06-22 21:40:19 +02:00
Zahary Karadjov 14274587cf More user-friendly logging during mainchain monitoring 2020-06-22 17:30:04 +03:00
Eugene Kabanov 47eaaa7696
Fix connection workers race. (#1204) 2020-06-21 18:49:48 +02:00
Jacek Sieka a661ecbae1
bump libp2p (#1209) 2020-06-21 18:45:28 +02:00