vacp2p
/
nim-libp2p-experimental
mirror of https://github.com/vacp2p/nim-libp2p-experimental.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
573 Commits 92 Branches 0 Tags 44 MiB
Commit Graph

403 Commits

Author SHA1 Message Date
Zahary Karadjov 77e866d29a
Address review comments 2019-12-09 19:17:29 +02:00
Zahary Karadjov 454f658ba8
Fixes and tweaks related to the beacon node integration 
* Bugfix: Dialing an already connected peer may lead to crash

* Introduced a standard_setup module allowing to instantiate
  the `Switch` object in an easier manner.

* Added `Switch.disconnect(peer)`

* Trailing space removed (sorry about polluting the diff)
 2019-12-08 23:58:43 +02:00
Dmitriy Ryajov 5f6fcc3d90
extract public and private keys fields from peerid (#44) 
* extract public and private keys fields from peerid

* allow assigning a public key

* cleaned up TODOs

* make pubsub prefix a const

* public key should be an `Option`
 2019-12-07 10:36:39 -06:00
Dmitriy Ryajov e623e70e7b
PubSub (Gossip & Flood) Implementation (#36) 
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon. 

* add close event

* wip: gossipsub

* splitting rpc message

* making message handling more consistent

* initial gossipsub implementation

* feat: nim 1.0 cleanup

* wip: gossipsub protobuf

* adding encoding/decoding of gossipsub messages

* add disconnect handler

* add proper gossipsub msg handling

* misc: cleanup for nim 1.0

* splitting floodsub and gossipsub tests

* feat: add mesh rebalansing

* test pubsub

* add mesh rebalansing tests

* testing mesh maintenance

* finishing mcache implementatin

* wip: commenting out broken tests

* wip: don't run heartbeat for now

* switchout debug for trace logging

* testing gossip peer selection algorithm

* test stream piping

* more work around message amplification

* get the peerid from message

* use timed cache as backing store

* allow setting timeout in constructor

* several changes to improve performance

* more through testing of msg amplification

* prevent gc issues

* allow piping to self and prevent deadlocks

* improove floodsub

* allow running hook on cache eviction

* prevent race conditions

* prevent race conditions and improove tests

* use hashes as cache keys

* removing useless file

* don't create a new seq

* re-enable pubsub tests

* fix imports

* reduce number of runs to speed up tests

* break out control message processing

* normalize sleeps between steps

* implement proper transport filtering

* initial interop testing

* clean up floodsub publish logic

* allow dialing without a protocol

* adding multiple reads/writes

* use protobuf varint in mplex

* don't loose conn's peerInfo

* initial interop pubsub tests

* don't duplicate connections/peers

* bring back interop tests

* wip: interop

* re-enable interop and daemon tests

* add multiple read write tests from handlers

* don't cleanup channel prematurely

* use correct channel to send/receive msgs

* adjust tests with latest changes

* include interop tests

* remove temp logging output

* fix ci

* use correct public key serialization

* additional tests for pubsub interop
 2019-12-05 20:16:18 -06:00
Dmitriy Ryajov 903e79ede1
Feat/conn cleanup (#41) 
Backporting proper connection cleanup from #36 to align with latest chronos changes.

* add close event

* use proper varint encoding

* add proper channel cleanup in mplex

* add connection cleanup in secio

* tidy up

* add dollar operator

* fix tests

* don't close connections prematurely

* handle closing streams properly

* misc

* implement address filtering logic

* adding pipe tests

* don't use gcsafe if not needed

* misc

* proper connection cleanup and stream muxing

* re-enable pubsub tests
 2019-12-03 22:44:54 -06:00
Dmitriy Ryajov 1df16bdbce set log level to trace - not enabled by default 2019-12-02 18:43:21 -06:00
Zahary Karadjov f9eed172d4 Don't handle Exception 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov 26366f8b56 use `Exception` to emulate `getCurrentException()` 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov a5b051a14f fix: use exception `msg` field in logs 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov 9e437a681b fix: revert to proper exception handling 2019-12-03 01:26:59 +02:00
cheatfate dde8c01448
Workaround for https://github.com/nim-lang/Nim/issues/12764 . 
Remove usage of getCurrentException().
 2019-11-28 20:58:56 +02:00
Dmitriy Ryajov fe3c1d45bb misc: removing unneded accessors 2019-11-06 12:13:21 -06:00
Ștefan Talpalaru f3fc763895
daemonapi: direct p2pd's output to parent's streams 2019-11-05 21:16:56 +01:00
cheatfate 76fee97724
Fix MultiAddress.init(PeerID) compilation problem. 2019-11-04 20:57:17 +02:00
cheatfate 833e8c9406
Fix overflow problem. 
Fix compilation warnings.
 2019-10-29 20:51:48 +02:00
Dmitriy Ryajov 400218ba01 fixing remaining overflows in BE/LE 2019-10-29 11:02:28 -06:00
Dmitriy Ryajov 541f0f2a41 avoid buffer overflow on BE/LE conversions 2019-10-29 10:10:19 -06:00
cheatfate cc0d45c482
Add protoAddress() utility procedure. 2019-10-29 15:38:37 +02:00
Dmitriy Ryajov 21af081066 remove init in favor of upstream init 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a4431a86a1 fixing windows build issue 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0b77c8c6e3 add init from TransportAddress to multiaddress 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ae7b354819 unused directory 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9e16fd020e use PeerID `==` for comparisons 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 018da65708 compare peer string ids, rather than keys 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3194e3ac5b fixes after building with 0.20.2 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bb277d4b56 readLp should warn on ignored exceptions 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 464d75d159 minor style fixes 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2066e81658 set default timeout to 10 secs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1d4b51413e option to allow triggering own handlers on publish 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 37d7a03fba use a timed cache in floodsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e98ec9fd49 use doAssert for release asserts 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f190c155d3 don't throw on missing peer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3b9d34116d decrease floodsub traffic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f9a04dbb9d todo: poll should allow for timeouts 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cfd31c5251 properly handle empty messages 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6cfbf2c124 don't send messages to self 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a01d1db6a0 prperly close and chronos connections 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 968fd85d49 properly exit readLoop on error/stream end 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 34d1a641de cleanup/test pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f8872dd51c make log topics consistent 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3c5b77affa correctly identify incoming connection 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 68eb142a7c end multiplex read loop on exception 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cc2072d290 always cleanup connection on exit 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c7606ce2eb readMsg should never return an empty buffer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f7e5c8ee30 bring back commented tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bc46a76029 connection should raise on invalid varint 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 15bae7bec5 use `result` instead of buffer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f5508be544 yield to chronos in mplex loop 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b5c4eaf9b8 don't use result in oneliners 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 06b759e049 review cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5ae37cac98 no more buffer resets on len prefixed reads 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov dd3f85965b should not handle transport exceptions 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 99869aa8b8 don't reset buffer on exception 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6baf089bb4 move tomultiaddr to wire.nim 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f3afe0a6ce feat: allow ussage of 0 addrs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9862064234 changed copyright year 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5b3f93ba1c feat: allow multiple handlers per topic in pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2f31fc6940 misc: TODO for timeouts and message limit in mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b270515bb3 feat: make private/public keys Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 663ce6c589 misc: nimpretty 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 054085620c logging: switch debug for trace in most cases 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 011df568b7 finished integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3eb0cdd5f7 misc 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9bb892de69 wip: integrating and testing secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 27e4516796 fix breaking test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b47dc89589 wip: integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ea142f0e6d proxy connection through secio 2019-10-11 08:15:24 +09:00
cheatfate 2d00f6a6db wip: integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f2fb92d1af properly close transport 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c98e8929bc reverting change 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f39a5cf5e4 temporary fix, until pub/priv keys are Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ef89e98479 renaming channel 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 88079a2864 incomming connection handling flow and tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 68d50a97f8 properly initialize hashsets 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9f3b80b60c got pubsub working without signing 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 80267e81ec exit main loop correctly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8920cd7d60 misc: pubsub/floodsub and logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 4d9444afe9 proper server startup sequence 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5f2d944545 add muxer handler and properly set stream peerinfo 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 32b74c5d1c add logging context 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 875ec346e6 adding bool suport to protobuf methods 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 003b72ec27 adding bool 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 67e10d0747 set mplex channel peerInfo correctly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 4f158d4e83 adding pubsub interface to switch 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 41aad2e7e6 make peerId an Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 177eb71ffa wip: floodsub initial implementation 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 74d2aea63d adding chronicles topics 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 827a8caba6 wip: modeling floodsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bb648c87e1 wip: modeling secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 525646dcc1 handle secure managers correctly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov fc36de21c1 handle exceptions correctly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2febd47e0d don't complete futures twise 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov d27ea5d228 whip initial secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 435c69633f adding chronicles logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5bc8e7e7b1 remap exceptions 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 116051208a make patterns runtime only 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 93a9fd203a fix: multistream tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e31966b6f8 fix: switch, with identify and mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 701e048ee6 add debug logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 54d740949e debug logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0c416e757e make gcsafe 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ddd266f584 small helper to aid in debugging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b5dcfa9bc4 first attempt to proper channel cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 65ce1a93fc prevent interleaved channel data 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8c76799d9e make writes sequential 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e53c87e197 reworked to make msg reading sequential 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c7fd08e002 use sane defaults 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e5b782f094 mplex testing/interop 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 100f6220b3 dont clash with MultiCodec 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6086038b01 raise correct LP exception on incomplete data 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ba843f4091 wip: interop testing 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 13ca6d253d fix: multistream tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 962f58b436 avoid wrapping in several futures 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 13837619ed plaintext security 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8338a16aab testing mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b7f999d316 fixing several issues found while testing 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov dee46c4d37 remove unneded var 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 14d9150bbb moving modules to apropriate directories 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2f402d68e1 docs/wording 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 60245a065d fix: don't call conn handler on dials 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0f52a6e798 fix: switch and tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6f8de062bb fix: multistream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5237fc8b12 test multiple streams with mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 746987156d reworked switch 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0b784c5b58 don't pass stream handler through contructor 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3cd19ddc47 change root object from ref to obj 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f8a942ed67 make peerinfo an optional type 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9889bd9cbf return proto string from select, instead of bool 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cc595f7947 allow selecting one of many protos in identify 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a7e5fde6f7 check identify request against known pubkey 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ec351cc2b0 misc: cleanup mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6ce2782e5c add missing test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 80c68206f7 documentation 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9b485b3082 feat: half closed channels 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6058a3fc69 split mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1b02c5e4e4 don't block main loop 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 36917c5adc disable switch tests temporarily 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a0aaa29739 wip: mplex and mplex tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cc26d4e040 prevent already closed errors 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8549ce76bb add init proc to bufferedstream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 917e0553e1 fix: tests and docs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 96cd7bcf50 wip: mplex tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b26d1ac23a cosmetic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 93bd7f623c remove generic constructor 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9bc7043f48 export read size const 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cae4910427 add custom new proc 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 40e75a7635 remove useless methods 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 4edd2c9f8a mplex move everything to one file 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ad1eaffdd6 make writeHandler async 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov d63e0c003b add proper pragmas 2019-10-11 08:15:24 +09:00