Commit Graph

120 Commits

Author SHA1 Message Date
Giovanni Petrantoni be0375f840 fix up merge 2020-07-04 11:51:09 +09:00
Giovanni Petrantoni afb5cb88f9 Merge remote-tracking branch 'origin/publish-backporting' into gossip-one-one 2020-07-04 11:42:16 +09:00
Giovanni Petrantoni 8d0709ce5d merge new pubsubpeer, copy 1.1 to new module 2020-07-04 11:29:37 +09:00
Giovanni Petrantoni 0d0309a601 better diagnostics 2020-07-03 09:28:20 +09:00
Giovanni Petrantoni 27d22d534d simplify rebalanceMesh (following spec) and make it finally reach D_high 2020-07-02 23:07:17 +09:00
Giovanni Petrantoni 5e4934bcce Merge branch 'master' into publish-backporting 2020-07-02 17:44:23 +09:00
Giovanni Petrantoni 4f5c97e27c remove topics from pubsubpeer (was unused) 2020-07-01 17:44:26 +09:00
Giovanni Petrantoni d9e0ca6091 getGossipPeers fixes 2020-07-01 17:26:42 +09:00
Giovanni Petrantoni a976c22dae triage publish nil peers (issue is on master too but just hidden behind a if/in) 2020-07-01 16:54:28 +09:00
Giovanni Petrantoni ec00c7fc50
Peer resultification and defect only (#245)
* Peer resultification and defect only

* Fixing some tests

* test fixes

* Rename peer into peerid

* better result error message in identify

* further merge fixes
2020-07-01 08:25:09 +02:00
Giovanni Petrantoni f49e59cb4e improve fanout replenish behavior further 2020-06-30 11:36:47 +09:00
Giovanni Petrantoni 509d84f03d restore trace vs debug in gossip 2020-06-30 11:28:45 +09:00
Giovanni Petrantoni 9d33390239 restore utility module import 2020-06-30 00:17:58 +09:00
Dmitriy Ryajov c788a6a3c0
Cleanup resources (#246)
* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises
2020-06-29 09:15:31 -06:00
Giovanni Petrantoni 680dcfd9cb cleanups 2020-06-30 00:10:11 +09:00
Giovanni Petrantoni 3a657135dc Merge branch 'master' into publish-backporting 2020-06-29 22:02:58 +09:00
Giovanni Petrantoni 3dd28adc9f restore old replenishFanout 2020-06-29 10:46:59 +09:00
Giovanni Petrantoni 2356f6ab68 make sure to not remove peers from gossip table 2020-06-29 10:45:06 +09:00
Jacek Sieka aa6756dfe0
allow message id provider to be specified (#243)
* don't send public key in message when not signing (information leak)
* don't run rebalance if there are peers in gossip (see #242)
* don't crash randomly on bad peer id from remote
2020-06-28 09:56:38 -06:00
Giovanni Petrantoni b7382eaf25 exclude non gossip from fanout 2020-06-28 10:55:44 +09:00
Giovanni Petrantoni 8e2c9301a0 use g.peers for fanout (todo: don't include flood peers) 2020-06-28 10:53:43 +09:00
Giovanni Petrantoni ccb323ad0f big publish refactor, replenish and balance 2020-06-27 20:24:29 +09:00
Giovanni Petrantoni dc3c568dc0 wip... lvalues don't work properly sadly... 2020-06-27 14:59:48 +09:00
Giovanni Petrantoni 1a2f336eb5 use var semantics to optimize table access 2020-06-27 13:35:07 +09:00
Giovanni Petrantoni ab8d7cc968 make sure to waitsub in sparse test 2020-06-26 16:24:56 +09:00
Giovanni Petrantoni 03801fcc52 small fixes, publish still half broken 2020-06-26 16:06:08 +09:00
Giovanni Petrantoni ec5c0b867d add floodPublish logic 2020-06-26 12:39:45 +09:00
Giovanni Petrantoni cf51da00b2 take into account explicit direct peers in publish 2020-06-26 12:31:08 +09:00
Giovanni Petrantoni 98e4bb6769 add a way to pass parameters to gossip 2020-06-25 21:14:22 +09:00
Dmitriy Ryajov 7a95f1844b
Concurrent dials (#238)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* add secureconn constructor

* close in the correct order

* concurent dial lock and track in/out conns better

* make tests pass

* add todo comment

* disconect peers that open too many connections

* wip

* do connection and muxer tracking in one place

* prevent nil pointer in observers

* drop connections when peers is over max

* prevent channel leaks

* don't use closure to handle channel
2020-06-24 09:08:44 -06:00
Giovanni Petrantoni 08ed2a7974 prune by score (score is stub still) 2020-06-24 12:43:35 +09:00
Giovanni Petrantoni cc3a0c29c0 validated always, even explicit peers 2020-06-24 12:19:59 +09:00
Giovanni Petrantoni eea48f8bd5 fix encodePrune 2020-06-23 17:05:13 +09:00
Giovanni Petrantoni 7748b8c5c9 PeerInfo and ControlPrune protocols 2020-06-23 17:01:21 +09:00
Giovanni Petrantoni 99b982c21d explicit peering part 3 2020-06-23 17:01:05 +09:00
Giovanni Petrantoni f8292f7086 explicit peering part 2 2020-06-23 15:56:07 +09:00
Giovanni Petrantoni 9eb240fb7d add 1.1 protocol to gossip 2020-06-23 11:49:14 +09:00
Giovanni Petrantoni 3568b073fd allow multiple codecs per protocol (without breaking things) 2020-06-23 11:48:11 +09:00
Giovanni Petrantoni 7852c6dd0f
Noise and eth2/nbc fixes (#226)
* Remove noise padding payload (spec removed it)

* add log scope in secure

* avoid defect array out of range in switch secure when "na"

* improve identify traces

* wip noise fixes

* noise protobuf adjustments (trying)

* add more debugging messages/traces, improve their actual contents

* re-enable ID check in noise

* bump go daemon tag version

* bump go daemon tag version

* enable noise in daemonapi

* interop testing, (both secio and noise will be tested)

* azure cache bump (p2pd)

* CI changes

- Travis: use Go 1.14
- azure-pipelines.yml: big cleanup
- Azure: bump cache keys
- build 64-bit p2pd on 32-bit Windows
- install both Mingw-w64 architectures

* noise logging fixes

* alternate testing between noise and secio

* increase timeout to avoid VM errors in CI (multistream tests)

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2020-06-20 19:56:55 +09:00
Dmitriy Ryajov 7a1c1c2ea6
fixing some key not found exceptions (#231) 2020-06-19 15:19:07 -06:00
Dmitriy Ryajov 5b28e8c488
Cleanup lpstream, Connection and BufferStream (#228)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* don't use delegation in connection

* move connection out to own file

* don't breakout on reset

* make sure to call close on secured conn

* add lpstream tracing

* don't breackdown by conn id

* fix import

* remove unused lable

* reset  connection on exception

* add additional metrics for skipped messages

* check for nil in secure.close
2020-06-19 11:29:43 -06:00
Dmitriy Ryajov 719744f46a
Small fixes (#230)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* make sure sending doesn't fail

* add `contains`

* review comment from prev pr
2020-06-19 11:29:25 -06:00
Dmitriy Ryajov fe828d87d8
count published messages (#224) 2020-06-16 22:14:02 -06:00
Dmitriy Ryajov 9d9f793b4f
add metrics for sent messages by topic and peer (#220) 2020-06-15 17:39:03 -06:00
Dmitriy Ryajov 7cb6c81159
Don't modify iterables while iterating them (#219)
* don't modify iterables while iterating

* assert handlers to properly close connections
2020-06-15 12:30:09 -06:00
Oskar Thorén f97e2deec7
Make methods in FloodSub, GossipSub public (#216)
Similar to https://github.com/status-im/nim-libp2p/pull/193 but doing it for all
methods to avoid this issue in PubSub in the future. Got hit by this behavior
again in rpcHandler and took me a while to figure out.

See https://github.com/oskarth/nim-private-method-skipping-case/ for minimal
repro
2020-06-12 17:54:12 -06:00
Dmitriy Ryajov ac04ca6e31
make sure keys exist and more metrics (#215) 2020-06-11 20:20:58 -06:00
Dmitriy Ryajov 55a294a5c9
better pubsub metrics (#214) 2020-06-11 12:09:34 -06:00
Dmitriy Ryajov 6b196ad7b4
remove pubsub peer on disconnect (#212)
* remove pubsub peer on disconnect

* make sure lock is aquired

* add $

* count upgrades/dials/disconnects
2020-06-11 08:45:59 -06:00
Viktor Kirilov 1afec627c2
proper name for topics so that we can filter dynamically using chronicles (#210)
* proper name for topics so that we can filter dynamically using chronicles

* lowercase
2020-06-10 10:48:01 +02:00