Commit Graph

141 Commits

Author SHA1 Message Date
Giovanni Petrantoni 84cc85b536 Merge branch 'master' into gossip-one-one 2020-09-08 18:03:13 +09:00
Jacek Sieka c1856fda53
simplify and unify logging (#353)
* use short format for logging peerid
* log peerid:oid for connections
2020-09-06 10:31:47 +02:00
Jacek Sieka 9b815efe8f
gossipsub: don't subscribe to floodsub also (#352) 2020-09-04 22:53:03 +02:00
Jacek Sieka 6d91d61844
small cleanups & docs (#347)
* simplify gossipsub heartbeat start / stop
* avoid alloc in peerid check
* stop iterating over seq after unsubscribing item (could crash)
* don't crash on missing private key with enabled sigs (shouldn't happen
but...)
2020-09-04 18:31:43 +02:00
Eugene Kabanov 0b85192119
Remove asyncCheck from codebase. (#345)
* Remove asyncCheck from codebase.

* Replace all `discard` statements with new `asyncSpawn`.

* Bump `nim-chronos` requirement.
2020-09-04 18:30:45 +02:00
Jacek Sieka 5819c6a9a7
gossipsub / floodsub fixes (#348)
* mcache fixes

* remove timed cache - the window shifting already removes old messages
* ref -> object
* avoid unnecessary allocations with `[]` operator

* simplify init

* fix several gossipsub/floodsub issues

* floodsub, gossipsub: don't rebroadcast messages that fail validation
(!)
* floodsub, gossipsub: don't crash when unsubscribing from unknown
topics (!)
* gossipsub: don't send message to peers that are not interested in the
topic, when messages don't share topic list
* floodsub: don't repeat all messages for each message when
rebroadcasting
* floodsub: allow sending empty data
* floodsub: fix inefficient unsubscribe
* sync floodsub/gossipsub logging
* gossipsub: include incoming messages in mcache (!)
* gossipsub: don't rebroadcast already-seen messages (!)
* pubsubpeer: remove incoming/outgoing seen caches - these are already
handled in gossipsub, floodsub and will cause trouble when peers try to
resubscribe / regraft topics (because control messages will have same
digest)
* timedcache: reimplement without timers (fixes timer leaks and extreme
inefficiency due to per-message closures, futures etc)
* timedcache: ref -> obj
2020-09-04 08:10:32 +02:00
Jacek Sieka cd1c68dbc5
avoid send deadlock by not allowing send to block (#342)
* avoid send deadlock by not allowing send to block

* handle message issues more consistently
2020-09-01 09:33:03 +02:00
Giovanni Petrantoni e225f3cf5a Merge branch 'master' into gossip-one-one 2020-08-24 17:41:45 +09:00
Dmitriy Ryajov d3182c4dba
No raise send (#339)
* dont raise in send

* check that the lock is acquire on release
2020-08-20 20:50:33 -06:00
Giovanni Petrantoni 3b8e85c792 Merge branch 'master' into gossip-one-one 2020-08-19 18:41:38 +09:00
Jacek Sieka b12145dff7
avoid crash when subscribe is received (#333)
...by making subscribeTopic synchronous, avoiding a peer table lookup
completely.

rebalanceMesh will be called a second later - it's fine
2020-08-17 12:10:22 +02:00
Giovanni Petrantoni eebaeb779f Merge branch 'master' into gossip-one-one 2020-08-17 11:44:23 +09:00
Giovanni Petrantoni afaa7f2a84 finishup with score and ihave budget 2020-08-17 01:20:50 +09:00
Jacek Sieka ab864fc747
logging cleanups and small fixes (#331) 2020-08-15 21:50:31 +02:00
Giovanni Petrantoni 1661344e17 oversub prune score based, finish outbound quota 2020-08-15 16:25:29 +09:00
Giovanni Petrantoni 69ff05a4f4 outbound mesh quota, internal tests fixing 2020-08-15 12:51:51 +09:00
Giovanni Petrantoni 309a4998c8 Adaptive gossip dissemination 2020-08-15 11:57:22 +09:00
Giovanni Petrantoni 3e8349b918 IWANT cap/budget 2020-08-14 17:07:26 +09:00
Giovanni Petrantoni c6fc8dee54 backoff time management 2020-08-14 11:59:19 +09:00
Giovanni Petrantoni ce61d84db4 add an hard cap to PX 2020-08-13 21:21:17 +09:00
Giovanni Petrantoni 585cfc5996 implement peer exchange graft message 2020-08-13 20:41:54 +09:00
Giovanni Petrantoni 05463873a2 restore explicit peering 2020-08-12 18:25:57 +09:00
Giovanni Petrantoni 296ca14133 rebalance improvements and opportunistic grafting 2020-08-12 16:30:03 +09:00
Giovanni Petrantoni f7fee2ca97 further merge fixes 2020-08-12 12:04:54 +09:00
Giovanni Petrantoni c8954619ab post merge - builds gossip tests 2020-08-12 11:17:36 +09:00
Giovanni Petrantoni 90a33c0a29 Merge branch 'master' into gossip-one-one 2020-08-12 10:57:32 +09:00
Dmitriy Ryajov b76b3e0e9b
Rework pubsub (#322)
* move pubsub of off switch, pass switch into pubsub

* use join on lpstreams

* properly cleanup up failed peers

* fix tests

* fix peertable hasPeerId

* fix tests

* rework sending, remove helpers from pubsubpeer, unify in broadcast

* further split broadcast into send

* use send where appropriate

* use formatIt

* improve trace

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-08-11 18:05:49 -06:00
Giovanni Petrantoni 66c9cd2c8c score fixes 2020-08-08 18:27:57 +09:00
Giovanni Petrantoni 4dcc3b6dda Merge remote-tracking branch 'origin/master' into gossip-one-one 2020-08-08 16:53:14 +09:00
Giovanni Petrantoni 9bbe5e4841
Fix subclass calls to handleDisconnect (#314)
* Fix subclass calls to handleDisconnect

* add peer ID to nil peer debug message
2020-08-06 11:12:52 +09:00
Giovanni Petrantoni 5c986cf657
Fix build, add some raises (#315)
* Fix build, add some raises

* wip

* wip more raises

* missing exc object in mplex

* proper lifetime for subscribePeer

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2020-08-05 19:30:57 -06:00
Ștefan Talpalaru bd5d43874a
more expensive metrics (#312) 2020-08-05 14:02:26 +02:00
Ștefan Talpalaru 843d32f8db
put expensive metrics under a Nim define (#310) 2020-08-04 17:27:59 -06:00
Dmitriy Ryajov 980764774e
pubsub timeouts tuning (#295)
* add finegrained timeouts to pubsub

* use 10 millis timeout in tests

* finalization

* revert timeouts

* use `atEof` for reads

* adjust timeouts and use atEof for reads

* use atEof for reads

* set isEof flag

* no backoff for pubsub streams

* temp timer increase, make macos finalize

* don't call `subscribePeer` in libp2p anymore

* more traces

* leak tests

* lower timeouts

* handle exceptions in control message

* don't use `cancelAndWait`

* handle exceptions in helpers

* wip

* don't send empty messages

* check for leaks properly

* don't use cancelAndWait

* don't await subscribption sends

* remove subscrivePeer calls from switch

* trying without the hooks again
2020-08-02 23:20:11 -06:00
Jacek Sieka e655a510cd
misc cleanups (#303) 2020-08-02 12:22:49 +02:00
Giovanni Petrantoni 7946743ae5 wip 2020-07-29 09:41:34 +09:00
Giovanni Petrantoni a34eee7ed4 pruning improvements 2020-07-28 14:09:37 +09:00
Dmitriy Ryajov f7fdf31365
Pubsub lifetime (#284)
* lifecycle hooks

* tests

* move trace after closed check

* restore 1 second heartbeat

* await close event

* fix tests

* print direction string

* more trace logging

* add pubsub monitor

* add log scope

* adjust idle timeout

* add exc.msg to trace
2020-07-27 13:33:51 -06:00
Giovanni Petrantoni f73cf36afb add a handleConnect in subscribeTopic 2020-07-22 11:00:02 +09:00
Giovanni Petrantoni f122ec9dbf fix grafted defect 2020-07-22 10:19:53 +09:00
Giovanni Petrantoni aded78f259 don't delete while iterating in score update 2020-07-21 13:37:06 +09:00
Giovanni Petrantoni b61f1da9a8 fixes 2020-07-21 11:47:46 +09:00
Giovanni Petrantoni 884d05cbc2 score wip 2020-07-21 11:25:14 +09:00
Giovanni Petrantoni 0776cc77e7 Merge branch 'master' into gossip-one-one 2020-07-21 11:02:02 +09:00
Giovanni Petrantoni 3b088f8980
Fix some unsubscribe issues and add unsubscribeAll helper (#282)
* Fix some unsub issues and add unsuball helper

* batch sendprune in unsubscribe methods

* add unsubscribeAll for floodsub
2020-07-20 10:16:13 -06:00
Giovanni Petrantoni afe17ca2c1 score wip 2020-07-20 18:20:22 +09:00
Giovanni Petrantoni b8a519884e tables troubleshooting 2020-07-20 16:20:49 +09:00
Giovanni Petrantoni d9fa9e2e84 wip 2020-07-19 12:37:45 +09:00
Giovanni Petrantoni 6af7909378 fix internal gossip tests 2020-07-18 21:00:44 +09:00
Giovanni Petrantoni 13c87a76c0 pubsubpeer, chronicles explicit override 2020-07-18 18:09:40 +09:00