Commit Graph

223 Commits

Author SHA1 Message Date
Jacek Sieka 4479c0a9f1
fix some libp2p channel hangs (#2032)
* fix some libp2p channel hangs

* make sure no concurrent reads are issues by awaiting cancellations
* bumps with fixes

* merge bump
2020-11-17 16:07:05 +01:00
tersec 9e716b32bd
address some XDeclaredButNotUsed hints (#2028) 2020-11-17 11:14:53 +01:00
Jacek Sieka a6b188bfd4
misc fixes (#2027)
* log when database is loading (to avoid confusion)
* generate network keys later during startup
* fix quarantine not scheduling chain of parents for download and
increase size to one epoch
* log validator count, enr and peerid more clearly on startup
2020-11-16 20:15:43 +01:00
tersec 21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports (#2025) 2020-11-16 17:10:51 +01:00
cheatfate 19707e89c2 Change random network key log line and include PeerID. 2020-11-16 17:41:23 +02:00
tersec a0621d52f6
bump remaining rc.0 spec ref (#2003) 2020-11-12 13:58:41 +00:00
Jacek Sieka 43c6c5a0b3
Gossip message id should decompress data (#1998)
* also fix missing return on empty message
2020-11-12 11:45:28 +01:00
Jacek Sieka c5710d97fc
default to gossipsub 1.1 (#1992) 2020-11-11 13:30:18 +01:00
Zahary Karadjov 389c11743a
Review TODO items and self-assign the most important ones 2020-11-10 20:41:04 +02:00
tersec 7eaae89263
keep AllTests-mainnet.md consistent; ensure don't send overly large messages aren't sent (#1973) 2020-11-09 18:11:09 +00:00
Zahary Karadjov bd1047b715 0.6.0 release fixes
* Updated README
* No double v in nimbus_beacon_node --version
* No md5 checksums
2020-11-09 17:09:49 +02:00
Sacha Saint-Leger 36b54fe091 Update eth2_network.nim
More informative and friendlier log message (from the user's perspective). Also helps avoid the potential for confusion with the validator key.
2020-11-03 23:23:32 +02:00
Jacek Sieka 937decb0e9
Revert "Gossip11 post audit (#1917)" (#1947)
This reverts commit 63173ab2c1.

It appears the cluster is having trouble staying connected - since the culprit is unknown, this is a first step on the way to what was stable.

Notably, this does not fully revert libp2p itself, merely the gossip version.
2020-11-03 11:45:40 +01:00
Giovanni Petrantoni 63173ab2c1
Gossip11 post audit (#1917)
* bump libp2p gossip

* use gossip 1.1

* use proper libp2p commit hash

* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
Jacek Sieka ee2ebc96a8
move rpc api to own folder, mimic upstream structure (#1905)
also implements a few more endpoints
2020-10-27 10:00:57 +01:00
Kim De Mey 756b62fdf5
Bump nim-eth to get UseDiscv51 flag (#1895)
* Bump nim-eth to get UseDiscv51 flag

* Switch medalla to discovery v5.1, other targets to v5.0

* Bump nim-eth for better discv5.1 logging

* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
tersec decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling (#1898)
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling

* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
tersec a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation (#1893) 2020-10-20 12:31:20 +00:00
tersec e87e0ef179
update to 1.0.0-rc.0 message ID; unexport funcs; enable test in mainnet; remove unused subscribe-with-nondummy-handler function; fix gossip received metric (#1884) 2020-10-20 08:54:11 +00:00
Zahary Karadjov e69af00e3a Don't penalize flooding peers; Just rate limit them 2020-10-15 20:15:51 +03:00
Zahary Karadjov 8a6281aad2 Simple cost model for sync requests; Penalize peers perfoming flooding or invalid requests 2020-10-15 20:15:51 +03:00
tersec 3ee2dd8da4
p2p-interface spec ref bump (except non-updated places) (#1862) 2020-10-12 14:37:14 +00:00
Zahary Karadjov 00a8a68671
Address #1695
Better error messages when the beacon node is asked to listen on a
reserved port (0) or an already taken one.
2020-10-09 16:39:03 +03:00
tersec b79e5f8af5
update nim-beacon-chain to nimbus-eth2 in beacon_chain/, ncli/, tests/, and README.md (#1843) 2020-10-08 19:02:05 +00:00
cheatfate f091c8d4df Add chronicles.formatIt for PublicKey.
Fix logs.
Rename checkFilePermissions to checkSensitiveFilePermissions.
2020-10-05 22:19:50 +03:00
cheatfate add22a20e1 Update local_testnet and simulation scripts to use netkey-file and insecure-netkey-password.
Add more logging
2020-10-05 22:19:50 +03:00
cheatfate e1182f8000 Add insecure password for automated testing.
Fix checkDataDir to run before setupLogging.
2020-10-05 22:19:50 +03:00
cheatfate 40f2b74f73 Add keystore management and interactive password handling. 2020-10-05 22:19:50 +03:00
cheatfate c5c788a9db Secure network key file and data directory. 2020-10-05 22:19:50 +03:00
Mamy Ratsimbazafy b57693ec0d
Logging update (#1795)
* Fix discovery log message trigger

* Bump chronicles - include https://github.com/status-im/nim-chronicles/pull/89 for better NOTICE/WARNING color
2020-10-03 08:35:45 +02:00
tersec 5e95fd7468
Revert "update to v0.12.3 message ID for Spadina launch (#1762)" (#1801)
This reverts commit a2270a5f27.
2020-10-02 19:50:21 +00:00
Mamy Ratsimbazafy 0280d6c73e
Revisiting log levels (#1788)
* Update log level - https://github.com/status-im/nim-beacon-chain/issues/1779 https://github.com/status-im/nim-beacon-chain/issues/1785

* Address review comments

* Document the logging strategy [skip ci]
2020-10-01 20:56:42 +02:00
tersec a2270a5f27
update to v0.12.3 message ID for Spadina launch (#1762)
* update to v0.12.3 message ID for Spadina launch

* remove base64 import
2020-09-28 17:07:10 +02:00
Kim De Mey 23bec99341
Let also discovery used listen-address cli option instead of always any address (#1658) 2020-09-27 22:00:24 +02:00
Jacek Sieka 7837646079
anonymize libp2p messages (#1756)
* anonymize libp2p messages

* bump
2020-09-25 18:40:30 +02:00
Eugene Kabanov 1bf8d3af33
Disconnect peers with low score. (#1747)
* Disconnect peers with low score.

* Change PeerScoreLow value.

* Add spec url for DisconnectionReason.
2020-09-25 15:43:45 +02:00
tersec f96ad87d28
switch another 50+ spec refs from v0.12.2 to v0.12.3 (#1749) 2020-09-25 11:52:50 +00:00
Jacek Sieka b3a9afa0b1
libp2p: limit max gossip writes (#1739)
* libp2p: limit max gossip writes

* bump
2020-09-24 19:03:17 +02:00
Jacek Sieka e1c177cdd1
bump libp2p (#1721)
gossipsub 1.1 can be enabled with -d:nbc_gossipsub_11
2020-09-22 19:34:34 +02:00
Eugene Kabanov 654b8d66bf
Peer management (#1707)
* addPeer() and addPeerNoWait() now returns PeerStatus, not bool.
Minor refactoring of PeerPool.
Fix tests.

* Refactor PeerPool.
Add lenSpace.
Add tests for lenSpace.
PeerPool.add procedures now return different error codes.
Fix SyncManager break/continue problem.
Fix connectWorker break/continue problem.
Refactor connectWorker and discoveryLoop.
Fix incoming/outgoing blocking problem.

* Refactor discovery loop.
Add checkPeer.

* Fix logic and compilation bugs.

* Adjust position of debugging log.

* Fix issue with maximum peers in PeerPool.
Optimize node record decoding.

* fix discoveryLoop.

* Remove aliases and fix tests using aliases.
2020-09-21 18:02:27 +02:00
Jacek Sieka fc10f5121a protect against data after initial request
spec requires that channel is closed

also, avoid some unnecessary futures
2020-09-18 21:34:07 +03:00
tersec e106549efe
keep REJECT/IGNORE of messages failing validation for libp2p scoring (#1676)
* keep REJECT/IGNORE status of messages failing validation for libp2p scoring

* fix test suite
2020-09-18 13:53:09 +02:00
Dmitriy Ryajov 2f89e2ab4e
drop subscribePeer, it's called from pubsub now (#1677) 2020-09-17 11:40:21 +02:00
Eugene Kabanov 6e463257f4
PeerPool fixes. (#1654)
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.

* Remove unused imports.

* Add test for a bug.

* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
2020-09-16 13:00:11 +03:00
Jacek Sieka c76305f824
fix some todo (#1645)
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
2020-09-14 14:50:03 +00:00
Jacek Sieka a087909683
fix req/resp protocol (#1621)
per spec, we must half-close request stream - not doing so may lead to
failure of the other end to start processing our request leading to
timeouts.

In particular, this fixes many sync problems that have been seen on
medalla.

* remove safeClose - close no longer raises
* use per-chunk timeouts in request processing
2020-09-10 21:40:09 +02:00
tersec d0de1a49a3
Fix some warnings and hints and partly revert #1610 (#1615)
* address some XDeclaredButNotUsed, ConvFromXtoItselfNotNeeded, and UnusedImport hints and warnings

* partly revert #1610
2020-09-08 11:32:43 +00:00
Jacek Sieka d584591ded
simplify libp2p logging (#1605)
and a few other small logging fixes
2020-09-06 10:39:25 +02:00
Jacek Sieka a7a279d615
add option to disable discv5 (#1509) 2020-08-24 13:52:06 +02:00
Jacek Sieka 22998fdfd4 avoid double deserialization
When blocks and attestations arrive, they are SSZ-decoded twice: once
for validation and once for processing. This branch enqueues the decoded
block directly for processing, avoiding the second, slow
deserialization.

* move processing of blocks and attestations to queue
* ...and out from beacon_node
* split attestation processing into attestations and aggregates
  * also updates metrics
* clean up logging to better follow the lifetime of gossip: arrival,
validation and processing
* drop attestations and aggregates if there are too many
* try to prioritise blocks and aggregates before single-validator
attestations
2020-08-21 11:46:25 +03:00