94 Commits

Author SHA1 Message Date
Giovanni Petrantoni
1661344e17 oversub prune score based, finish outbound quota 2020-08-15 16:25:29 +09:00
Giovanni Petrantoni
69ff05a4f4 outbound mesh quota, internal tests fixing 2020-08-15 12:51:51 +09:00
Giovanni Petrantoni
8932427a8b fix test for now 2020-08-12 17:03:42 +09:00
Giovanni Petrantoni
f7fee2ca97 further merge fixes 2020-08-12 12:04:54 +09:00
Giovanni Petrantoni
c8954619ab post merge - builds gossip tests 2020-08-12 11:17:36 +09:00
Giovanni Petrantoni
90a33c0a29 Merge branch 'master' into gossip-one-one 2020-08-12 10:57:32 +09:00
Dmitriy Ryajov
b76b3e0e9b
Rework pubsub (#322)
* move pubsub of off switch, pass switch into pubsub

* use join on lpstreams

* properly cleanup up failed peers

* fix tests

* fix peertable hasPeerId

* fix tests

* rework sending, remove helpers from pubsubpeer, unify in broadcast

* further split broadcast into send

* use send where appropriate

* use formatIt

* improve trace

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-08-11 18:05:49 -06:00
Giovanni Petrantoni
66c9cd2c8c score fixes 2020-08-08 18:27:57 +09:00
Giovanni Petrantoni
4dcc3b6dda Merge remote-tracking branch 'origin/master' into gossip-one-one 2020-08-08 16:53:14 +09:00
Dmitriy Ryajov
980764774e
pubsub timeouts tuning (#295)
* add finegrained timeouts to pubsub

* use 10 millis timeout in tests

* finalization

* revert timeouts

* use `atEof` for reads

* adjust timeouts and use atEof for reads

* use atEof for reads

* set isEof flag

* no backoff for pubsub streams

* temp timer increase, make macos finalize

* don't call `subscribePeer` in libp2p anymore

* more traces

* leak tests

* lower timeouts

* handle exceptions in control message

* don't use `cancelAndWait`

* handle exceptions in helpers

* wip

* don't send empty messages

* check for leaks properly

* don't use cancelAndWait

* don't await subscribption sends

* remove subscrivePeer calls from switch

* trying without the hooks again
2020-08-02 23:20:11 -06:00
Giovanni Petrantoni
7946743ae5 wip 2020-07-29 09:41:34 +09:00
Giovanni Petrantoni
880acd9dc1 Merge branch 'master' into gossip-one-one 2020-07-28 12:06:16 +09:00
Dmitriy Ryajov
f7fdf31365
Pubsub lifetime (#284)
* lifecycle hooks

* tests

* move trace after closed check

* restore 1 second heartbeat

* await close event

* fix tests

* print direction string

* more trace logging

* add pubsub monitor

* add log scope

* adjust idle timeout

* add exc.msg to trace
2020-07-27 13:33:51 -06:00
Giovanni Petrantoni
f74e693ea8 fix test utils generateNodes 2020-07-21 12:46:49 +09:00
Giovanni Petrantoni
b61f1da9a8 fixes 2020-07-21 11:47:46 +09:00
Giovanni Petrantoni
81f7413e87 Merge branch 'master' into gossip-one-one 2020-07-20 15:57:54 +09:00
Giovanni Petrantoni
c3af7659b0
Add more checks and fix some issues in gossip tests (#281) 2020-07-20 15:55:00 +09:00
Giovanni Petrantoni
d9fa9e2e84 wip 2020-07-19 12:37:45 +09:00
Giovanni Petrantoni
6af7909378 fix internal gossip tests 2020-07-18 21:00:44 +09:00
Giovanni Petrantoni
f708874170 finishup with params 2020-07-16 16:55:59 +09:00
Giovanni Petrantoni
66e5f41c38 Merge branch 'master' into gossip-one-one 2020-07-16 11:44:11 +09:00
Dmitriy Ryajov
f35b8999b3
some light cleanup for pub/gossip sub (#273)
* move peer table out to its own file

* move peer table

* cleanup `==` and add one to peerinfo

* add peertable

* missed equality check
2020-07-15 13:18:55 -06:00
Giovanni Petrantoni
b9759c0536 Merge branch 'master' into gossip-one-one 2020-07-15 23:22:59 +09:00
Giovanni Petrantoni
b166901f60 gossip internal tests fixes 2020-07-15 23:03:26 +09:00
Giovanni Petrantoni
d7bab37119
Fix gossip messages seqno according to spec (#253)
* Fix gossip messages seqno according to spec

* Add peers back to gossipsub table, slow down heartbeat

* Revert "Add peers back to gossipsub table, slow down heartbeat"

This reverts commit 01e2e62172a7793bb17f0eb8314e2faeb2682173.

* make seqno a threadvar, remove from peerinfo

* seqno refactor, into pubsub
2020-07-14 21:51:33 -06:00
Giovanni Petrantoni
8078fec0f0 Merge branch 'master' into gossip-one-one 2020-07-15 08:47:43 +09:00
Giovanni Petrantoni
92623eea53 gossipsub11 -> gossipsub 2020-07-15 08:27:58 +09:00
Ștefan Talpalaru
b8b0a2b4bc
CI: build binaries with TRACE & JSON logs (#268)
Also: remove unused imports.
2020-07-14 02:02:16 +02:00
Giovanni Petrantoni
fcda0f6ce1
PubSubPeer tables refactor (#263)
* refactor peer tables

* tests fixing

* override PubSubPeer equality

* fix pubsubpeer comparison
2020-07-13 15:32:38 +02:00
Dmitriy Ryajov
4c815d75e7
More gossip cleanup (#257)
* more cleanup

* correct pubsub peer count

* close the stream first

* handle cancelation

* fix tests

* fix fanout ttl

* merging master

* remove `withLock` as it conflicts with stdlib

* fix trace build

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-09 14:21:47 -06:00
Jacek Sieka
c720e042fc
clean up mesh handling logic (#260)
* gossipsub is a function of subscription messages only
* graft/prune work with mesh, get filled up from gossipsub
* fix race conditions with await
* fix exception unsafety when grafting/pruning
* fix allowing up to DHi peers in mesh on incoming graft
* fix metrics in several places
2020-07-09 11:16:46 -06:00
Giovanni Petrantoni
a7d90214ea Merge branch 'gossip-one-one' of github.com:status-im/nim-libp2p into gossip-one-one 2020-07-08 11:52:54 +09:00
Giovanni Petrantoni
0fbc0ec4b2 fix multistream 1.1 multi proto 2020-07-08 11:52:44 +09:00
Giovanni Petrantoni
38422ff4cb Merge branch 'master' into gossip-one-one 2020-07-08 11:39:33 +09:00
Dmitriy Ryajov
a52763cc6d
fix publishing (#250)
* use var semantics to optimize table access

* wip... lvalues don't work properly sadly...

* big publish refactor, replenish and balance

* fix internal tests

* use g.peers for fanout (todo: don't include flood peers)

* exclude non gossip from fanout

* internal test fixes

* fix flood tests

* fix test's trypublish

* test interop fixes

* make sure to not remove peers from gossip table

* restore old replenishFanout

* cleanups

* Cleanup resources (#246)

* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises

* restore trace vs debug in gossip

* improve fanout replenish behavior further

* cleanup stale peers more eaguerly

* synchronize connection cleanup and small refactor

* close client first and call parent second

* disconnect failed peers on publish

* check for publish result

* fix tests

* fix tests

* always call close

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-07 18:33:05 -06:00
Giovanni Petrantoni
57e6a95b9c Merge branch 'publish-backporting' into gossip-one-one 2020-07-07 21:54:07 +09:00
Giovanni Petrantoni
2d0e285810 Merge branch 'master' into publish-backporting 2020-07-07 20:58:58 +09:00
Jacek Sieka
d522537b19
reuse single RNG instance for all crypto key generation (#249)
* reuse single RNG instance for all crypto key generation

* use foolproof rng

* initRng -> newRng (because it's ref)

* fix test

* imports/exports, chat fix

* fix rsa

* imports and exports

* work around threadvar issue

* fixup

* mac workaround test
2020-07-07 13:14:11 +02:00
Giovanni Petrantoni
028b390c7f conditional enable gossip11 module 2020-07-04 11:58:35 +09:00
Giovanni Petrantoni
afb5cb88f9 Merge remote-tracking branch 'origin/publish-backporting' into gossip-one-one 2020-07-04 11:42:16 +09:00
Giovanni Petrantoni
5e4934bcce Merge branch 'master' into publish-backporting 2020-07-02 17:44:23 +09:00
Giovanni Petrantoni
4f5c97e27c remove topics from pubsubpeer (was unused) 2020-07-01 17:44:26 +09:00
Giovanni Petrantoni
ec00c7fc50
Peer resultification and defect only (#245)
* Peer resultification and defect only

* Fixing some tests

* test fixes

* Rename peer into peerid

* better result error message in identify

* further merge fixes
2020-07-01 08:25:09 +02:00
Giovanni Petrantoni
680dcfd9cb cleanups 2020-06-30 00:10:11 +09:00
Giovanni Petrantoni
3a657135dc Merge branch 'master' into publish-backporting 2020-06-29 22:02:58 +09:00
Jacek Sieka
aa6756dfe0
allow message id provider to be specified (#243)
* don't send public key in message when not signing (information leak)
* don't run rebalance if there are peers in gossip (see #242)
* don't crash randomly on bad peer id from remote
2020-06-28 09:56:38 -06:00
Giovanni Petrantoni
3f39eeb189 fix test's trypublish 2020-06-28 12:50:11 +09:00
Giovanni Petrantoni
31d79ebed8 fix flood tests 2020-06-28 11:58:12 +09:00
Giovanni Petrantoni
f98d2b635d internal test fixes 2020-06-28 11:03:33 +09:00
Giovanni Petrantoni
8e2c9301a0 use g.peers for fanout (todo: don't include flood peers) 2020-06-28 10:53:43 +09:00