Commit Graph

46 Commits

Author SHA1 Message Date
Dmitriy Ryajov e623e70e7b
PubSub (Gossip & Flood) Implementation (#36)
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon. 

* add close event

* wip: gossipsub

* splitting rpc message

* making message handling more consistent

* initial gossipsub implementation

* feat: nim 1.0 cleanup

* wip: gossipsub protobuf

* adding encoding/decoding of gossipsub messages

* add disconnect handler

* add proper gossipsub msg handling

* misc: cleanup for nim 1.0

* splitting floodsub and gossipsub tests

* feat: add mesh rebalansing

* test pubsub

* add mesh rebalansing tests

* testing mesh maintenance

* finishing mcache implementatin

* wip: commenting out broken tests

* wip: don't run heartbeat for now

* switchout debug for trace logging

* testing gossip peer selection algorithm

* test stream piping

* more work around message amplification

* get the peerid from message

* use timed cache as backing store

* allow setting timeout in constructor

* several changes to improve performance

* more through testing of msg amplification

* prevent gc issues

* allow piping to self and prevent deadlocks

* improove floodsub

* allow running hook on cache eviction

* prevent race conditions

* prevent race conditions and improove tests

* use hashes as cache keys

* removing useless file

* don't create a new seq

* re-enable pubsub tests

* fix imports

* reduce number of runs to speed up tests

* break out control message processing

* normalize sleeps between steps

* implement proper transport filtering

* initial interop testing

* clean up floodsub publish logic

* allow dialing without a protocol

* adding multiple reads/writes

* use protobuf varint in mplex

* don't loose conn's peerInfo

* initial interop pubsub tests

* don't duplicate connections/peers

* bring back interop tests

* wip: interop

* re-enable interop and daemon tests

* add multiple read write tests from handlers

* don't cleanup channel prematurely

* use correct channel to send/receive msgs

* adjust tests with latest changes

* include interop tests

* remove temp logging output

* fix ci

* use correct public key serialization

* additional tests for pubsub interop
2019-12-05 20:16:18 -06:00
Dmitriy Ryajov 903e79ede1
Feat/conn cleanup (#41)
Backporting proper connection cleanup from #36 to align with latest chronos changes.

* add close event

* use proper varint encoding

* add proper channel cleanup in mplex

* add connection cleanup in secio

* tidy up

* add dollar operator

* fix tests

* don't close connections prematurely

* handle closing streams properly

* misc

* implement address filtering logic

* adding pipe tests

* don't use gcsafe if not needed

* misc

* proper connection cleanup and stream muxing

* re-enable pubsub tests
2019-12-03 22:44:54 -06:00
Dmitriy Ryajov 1df16bdbce set log level to trace - not enabled by default 2019-12-02 18:43:21 -06:00
Zahary Karadjov f9eed172d4 Don't handle Exception 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov 26366f8b56 use `Exception` to emulate `getCurrentException()` 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov a5b051a14f fix: use exception `msg` field in logs 2019-12-03 01:26:59 +02:00
cheatfate dde8c01448
Workaround for https://github.com/nim-lang/Nim/issues/12764 .
Remove usage of getCurrentException().
2019-11-28 20:58:56 +02:00
cheatfate 833e8c9406
Fix overflow problem.
Fix compilation warnings.
2019-10-29 20:51:48 +02:00
Dmitriy Ryajov 9e16fd020e use PeerID `==` for comparisons 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 018da65708 compare peer string ids, rather than keys 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 464d75d159 minor style fixes 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2066e81658 set default timeout to 10 secs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1d4b51413e option to allow triggering own handlers on publish 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 37d7a03fba use a timed cache in floodsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f190c155d3 don't throw on missing peer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3b9d34116d decrease floodsub traffic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6cfbf2c124 don't send messages to self 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 968fd85d49 properly exit readLoop on error/stream end 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 34d1a641de cleanup/test pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9862064234 changed copyright year 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5b3f93ba1c feat: allow multiple handlers per topic in pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b270515bb3 feat: make private/public keys Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 663ce6c589 misc: nimpretty 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 054085620c logging: switch debug for trace in most cases 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 011df568b7 finished integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3eb0cdd5f7 misc 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9bb892de69 wip: integrating and testing secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b47dc89589 wip: integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ea142f0e6d proxy connection through secio 2019-10-11 08:15:24 +09:00
cheatfate 2d00f6a6db wip: integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f39a5cf5e4 temporary fix, until pub/priv keys are Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 88079a2864 incomming connection handling flow and tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 68d50a97f8 properly initialize hashsets 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9f3b80b60c got pubsub working without signing 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8920cd7d60 misc: pubsub/floodsub and logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 41aad2e7e6 make peerId an Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 177eb71ffa wip: floodsub initial implementation 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 74d2aea63d adding chronicles topics 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 827a8caba6 wip: modeling floodsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bb648c87e1 wip: modeling secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov d27ea5d228 whip initial secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 435c69633f adding chronicles logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e31966b6f8 fix: switch, with identify and mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 13837619ed plaintext security 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b7f999d316 fixing several issues found while testing 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 14d9150bbb moving modules to apropriate directories 2019-10-11 08:15:24 +09:00