210 Commits

Author SHA1 Message Date
Giovanni Petrantoni
63173ab2c1
Gossip11 post audit (#1917)
* bump libp2p gossip

* use gossip 1.1

* use proper libp2p commit hash

* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
Jacek Sieka
ee2ebc96a8
move rpc api to own folder, mimic upstream structure (#1905)
also implements a few more endpoints
2020-10-27 10:00:57 +01:00
Kim De Mey
756b62fdf5
Bump nim-eth to get UseDiscv51 flag (#1895)
* Bump nim-eth to get UseDiscv51 flag

* Switch medalla to discovery v5.1, other targets to v5.0

* Bump nim-eth for better discv5.1 logging

* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
tersec
decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling (#1898)
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling

* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
tersec
a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation (#1893) 2020-10-20 12:31:20 +00:00
tersec
e87e0ef179
update to 1.0.0-rc.0 message ID; unexport funcs; enable test in mainnet; remove unused subscribe-with-nondummy-handler function; fix gossip received metric (#1884) 2020-10-20 08:54:11 +00:00
Zahary Karadjov
e69af00e3a Don't penalize flooding peers; Just rate limit them 2020-10-15 20:15:51 +03:00
Zahary Karadjov
8a6281aad2 Simple cost model for sync requests; Penalize peers perfoming flooding or invalid requests 2020-10-15 20:15:51 +03:00
tersec
3ee2dd8da4
p2p-interface spec ref bump (except non-updated places) (#1862) 2020-10-12 14:37:14 +00:00
Zahary Karadjov
00a8a68671
Address #1695
Better error messages when the beacon node is asked to listen on a
reserved port (0) or an already taken one.
2020-10-09 16:39:03 +03:00
tersec
b79e5f8af5
update nim-beacon-chain to nimbus-eth2 in beacon_chain/, ncli/, tests/, and README.md (#1843) 2020-10-08 19:02:05 +00:00
cheatfate
f091c8d4df Add chronicles.formatIt for PublicKey.
Fix logs.
Rename checkFilePermissions to checkSensitiveFilePermissions.
2020-10-05 22:19:50 +03:00
cheatfate
add22a20e1 Update local_testnet and simulation scripts to use netkey-file and insecure-netkey-password.
Add more logging
2020-10-05 22:19:50 +03:00
cheatfate
e1182f8000 Add insecure password for automated testing.
Fix checkDataDir to run before setupLogging.
2020-10-05 22:19:50 +03:00
cheatfate
40f2b74f73 Add keystore management and interactive password handling. 2020-10-05 22:19:50 +03:00
cheatfate
c5c788a9db Secure network key file and data directory. 2020-10-05 22:19:50 +03:00
Mamy Ratsimbazafy
b57693ec0d
Logging update (#1795)
* Fix discovery log message trigger

* Bump chronicles - include https://github.com/status-im/nim-chronicles/pull/89 for better NOTICE/WARNING color
2020-10-03 08:35:45 +02:00
tersec
5e95fd7468
Revert "update to v0.12.3 message ID for Spadina launch (#1762)" (#1801)
This reverts commit a2270a5f270b328495572f51922631f5713600a4.
2020-10-02 19:50:21 +00:00
Mamy Ratsimbazafy
0280d6c73e
Revisiting log levels (#1788)
* Update log level - https://github.com/status-im/nim-beacon-chain/issues/1779 https://github.com/status-im/nim-beacon-chain/issues/1785

* Address review comments

* Document the logging strategy [skip ci]
2020-10-01 20:56:42 +02:00
tersec
a2270a5f27
update to v0.12.3 message ID for Spadina launch (#1762)
* update to v0.12.3 message ID for Spadina launch

* remove base64 import
2020-09-28 17:07:10 +02:00
Kim De Mey
23bec99341
Let also discovery used listen-address cli option instead of always any address (#1658) 2020-09-27 22:00:24 +02:00
Jacek Sieka
7837646079
anonymize libp2p messages (#1756)
* anonymize libp2p messages

* bump
2020-09-25 18:40:30 +02:00
Eugene Kabanov
1bf8d3af33
Disconnect peers with low score. (#1747)
* Disconnect peers with low score.

* Change PeerScoreLow value.

* Add spec url for DisconnectionReason.
2020-09-25 15:43:45 +02:00
tersec
f96ad87d28
switch another 50+ spec refs from v0.12.2 to v0.12.3 (#1749) 2020-09-25 11:52:50 +00:00
Jacek Sieka
b3a9afa0b1
libp2p: limit max gossip writes (#1739)
* libp2p: limit max gossip writes

* bump
2020-09-24 19:03:17 +02:00
Jacek Sieka
e1c177cdd1
bump libp2p (#1721)
gossipsub 1.1 can be enabled with -d:nbc_gossipsub_11
2020-09-22 19:34:34 +02:00
Eugene Kabanov
654b8d66bf
Peer management (#1707)
* addPeer() and addPeerNoWait() now returns PeerStatus, not bool.
Minor refactoring of PeerPool.
Fix tests.

* Refactor PeerPool.
Add lenSpace.
Add tests for lenSpace.
PeerPool.add procedures now return different error codes.
Fix SyncManager break/continue problem.
Fix connectWorker break/continue problem.
Refactor connectWorker and discoveryLoop.
Fix incoming/outgoing blocking problem.

* Refactor discovery loop.
Add checkPeer.

* Fix logic and compilation bugs.

* Adjust position of debugging log.

* Fix issue with maximum peers in PeerPool.
Optimize node record decoding.

* fix discoveryLoop.

* Remove aliases and fix tests using aliases.
2020-09-21 18:02:27 +02:00
Jacek Sieka
fc10f5121a protect against data after initial request
spec requires that channel is closed

also, avoid some unnecessary futures
2020-09-18 21:34:07 +03:00
tersec
e106549efe
keep REJECT/IGNORE of messages failing validation for libp2p scoring (#1676)
* keep REJECT/IGNORE status of messages failing validation for libp2p scoring

* fix test suite
2020-09-18 13:53:09 +02:00
Dmitriy Ryajov
2f89e2ab4e
drop subscribePeer, it's called from pubsub now (#1677) 2020-09-17 11:40:21 +02:00
Eugene Kabanov
6e463257f4
PeerPool fixes. (#1654)
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.

* Remove unused imports.

* Add test for a bug.

* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
2020-09-16 13:00:11 +03:00
Jacek Sieka
c76305f824
fix some todo (#1645)
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
2020-09-14 14:50:03 +00:00
Jacek Sieka
a087909683
fix req/resp protocol (#1621)
per spec, we must half-close request stream - not doing so may lead to
failure of the other end to start processing our request leading to
timeouts.

In particular, this fixes many sync problems that have been seen on
medalla.

* remove safeClose - close no longer raises
* use per-chunk timeouts in request processing
2020-09-10 21:40:09 +02:00
tersec
d0de1a49a3
Fix some warnings and hints and partly revert #1610 (#1615)
* address some XDeclaredButNotUsed, ConvFromXtoItselfNotNeeded, and UnusedImport hints and warnings

* partly revert #1610
2020-09-08 11:32:43 +00:00
Jacek Sieka
d584591ded
simplify libp2p logging (#1605)
and a few other small logging fixes
2020-09-06 10:39:25 +02:00
Jacek Sieka
a7a279d615
add option to disable discv5 (#1509) 2020-08-24 13:52:06 +02:00
Jacek Sieka
22998fdfd4 avoid double deserialization
When blocks and attestations arrive, they are SSZ-decoded twice: once
for validation and once for processing. This branch enqueues the decoded
block directly for processing, avoiding the second, slow
deserialization.

* move processing of blocks and attestations to queue
* ...and out from beacon_node
* split attestation processing into attestations and aggregates
  * also updates metrics
* clean up logging to better follow the lifetime of gossip: arrival,
validation and processing
* drop attestations and aggregates if there are too many
* try to prioritise blocks and aggregates before single-validator
attestations
2020-08-21 11:46:25 +03:00
Zahary Karadjov
2c19e3f8cd
[skip ci] Use GOSSIP_MAX_SIZE when snappy decoding in the inspector as well; Bumps 2020-08-19 14:33:52 +03:00
Zahary Karadjov
3433c77c35 Prevent Snappy decompression bombs 2020-08-19 10:13:04 +03:00
Dmitriy Ryajov
87f983c639 use split out pubsub 2020-08-17 17:24:36 +03:00
tersec
612881b95d
refactor topic (un)subscribing/validating to collate each (#1510)
* refactor topic (un)subscribing/validating to collate each

* fix comment

* tweak comment
2020-08-17 14:07:29 +02:00
tersec
af3355e0f8
create local testnet mode for eth2_network (#1494) 2020-08-12 14:16:59 +00:00
tersec
22c1ef5a8d
split subscribe into non-validating subscribe and addValidator (#1485)
* split subscribe into non-validating subscribe and addValidator

* stop exporting get_committee_assignments
2020-08-11 15:08:44 +00:00
Jacek Sieka
280e72f3c9
remove snappy RPC support (#1477)
removed in 0.12.2 - the flow, in particular when the other peer doesn't
support snappy, is hard to follow because of the trial-and-error
approach - removing it simplifies things and removes some of the
hard-to-read parts of the thunking etc
2020-08-10 15:18:17 +02:00
Jacek Sieka
936440fccd
use libp2p peer events to track peer (#1468)
this resolves some peer counting issues that were happening because the
lifetime future in PeerInfo was unreliable (multiple PeerInfo instances
existed per peer)

In addition, this solves another race condition: when connecting to a
peer and later dialling that protocol, it is not certain that the same
connection will be used if there's a concurrent incoming peer connection
ongoing - better not make too many assumptions about who sent statuses
when.
2020-08-10 12:58:34 +02:00
Jacek Sieka
585c410d90 remove randompeers
unused, requires importing `random` which we're trying to avoid
2020-08-10 11:48:33 +03:00
Dmitriy Ryajov
c5077af4bc
decreate amount of concurent dials (#1460) 2020-08-06 19:21:12 +00:00
Jacek Sieka
f4c16ed0db
eh cleanups (#1458)
current exception sometimes buggy in nim
2020-08-06 18:47:39 +00:00
Jacek Sieka
221f372dbc use peer id in a number of places 2020-08-05 19:34:59 +03:00
Jacek Sieka
d22a2cec2b
Start libp2p before writing ENR file (#1418)
this makes sure that all libp2p transports are open for business when
the file hits the ground
2020-08-03 19:35:27 +02:00