Commit Graph

259 Commits

Author SHA1 Message Date
Giovanni Petrantoni 4c6a123d31
Add chronos trackers and used them to sanitize resource disposal (#131)
* Add chronos trackers and used them to sanitize resource disposal

* Chronos trackers for transport tests wip

* No more chronos leaks in testtransport

* Make tcp transport and test more robust when closing

* Test async leaking tracking wip

* Fix a regression in wire connect

* Add chronos trackers to more tests and sanitize resource closure

* Wip fixing floodsub tests

* Floodsub wip

* Made floodsub basically deterministic, hit a nim bug with captures tho

* Wrap up floodsub tests refactor

* Wrapping up

* Add allFuturesThrowing utility

* Fix missing allFuturesThrowing in noise tests!

* Make tests green

* attempt fixing gossipsub failing cases

* Make sure to check also fanout in waitSub

* More verbose traces

* Gossipsub test improvments

* Refactor TcpTransport remove asyncCheck

* Add Connection trackers

* Add stricter connection tracking, wip mplex fix

* More asynccheck removal, in order to avoid connection leaks

* bump chronicles requirement

* Enable tracker dump to check CI output

* Wait for more futures in testmplex

* Remove tracker dump messages

* add tryAndWarn utility, fix mplex issue with go interop

* All allFuturesThrowing to directchat too

* make sure to cleanup on transport close
2020-04-21 10:24:42 +09:00
Jacek Sieka e8b33c64fa
secp: use upstream secp convenience api (#141)
* secp: use upstream secp convenience api
2020-04-17 12:51:13 +02:00
Dmitriy Ryajov 4ee1b4a66d connection closing tests 2020-04-07 12:16:59 -06:00
Giovanni Petrantoni e39bf0a4cf
Mplex: Add the ability to send any size payload (#123)
* Mplex: Add the ability to send any size payload

* Ensure size of coder header
2020-04-03 17:26:46 +02:00
Ștefan Talpalaru 7f8090b166
SecioConn.readMessage(): fix the exception types 2020-04-02 19:07:14 +02:00
Dmitriy Ryajov 5285f0d091
Fix/misc (#116)
* only check for payload size

* only subscribe if connection succeeded

* fix failing test

* check that the strem is active before openning

* msg type should not be > than 0x7

* fix tests

* check max against enum val
2020-03-29 08:28:48 -06:00
Dmitriy Ryajov 6bb4e91a39 fix tests 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov a92d616a3e fix tests 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov 3effb95f10 close underlying bufferstream in lpchannel 2020-03-28 09:29:43 -06:00
Giovanni Petrantoni c0a10ecea7
Disable new fragmentation tests (#119)
Merging for now to fix tests (virtually)
Could not repro locally yet
2020-03-27 10:47:30 +09:00
Giovanni Petrantoni 56392913f9
Allow to specify a read size in readLp (#117)
* Allow to specify a read size in readLp

Also fix secio fragments + test

* Revert readLp changes, use just `read` in test
2020-03-26 15:06:47 +09:00
Giovanni Petrantoni 1550bea1cc Support arbitrary stream write sizes in noise 2020-03-25 16:10:11 +09:00
cheatfate a2acdd7933
Disable unreliable provide cid test. 2020-03-24 10:05:11 +02:00
Giovanni Petrantoni c02fca25f8
Noise (#90)
* Start ChaCha20Poly1305 integration (BearSSL)

* Add Curve25519 (BearSSL) required operations for noise

* Fix curve mulgen iterate/derive

* Fix misleading header

* Add chachapoly proper test

* Curve25519 integration tests (failing, something is wrong)

* Add few converters, finish c25519 integration tests

* Remove implicit converters

* removed internal globals

* Start noise implementation

* Fix public() using proper bear mulgen

* Noise protocol WIP

* Noise progress

* Add a quick nim version of HKDF

* Converted hkdf to iterator, useful for noise

* Noise protocol implementation progress

* Noise progress

* XX handshake almost there

* noise progress

* Noise, testing handshake with test vectors

* Noise handshake progress, still wrong somewhere!

* Noise handshake success!

* Full verified noise XX handshake completed

* Fix and rewrite test to be similar to switch one

* Start with connection upgrade

* Switch chachapoly to CT implementations

* Improve HKDF implementation

* Use a type insted of tuple for HandshakeResult

* Remove unnecessary Let

* More cosmetic fixes

* Properly check randomBytes result

* Fix chachapoly signature

* Noise full circle (altho dispatcher is nil cursed)

* Allow nil aads in chachapoly routines

* Noise implementation up to running full test

* Use bearssl HKDF as well

* Directly use bearssl rng for curve25519 keys

* Add a (disabled/no CI) noise interop test server

* WIP on fixing interop issues

* More fixes in noise implementation for interop

* bump chronos requirement (nimble)

* Add a chachapoly test for very small size payloads

* Noise, more tracing

* Add 2 properly working noise tests

* Fix payload packing, following the spec properly (and not go version but
rather rust)

* Sanity, replace discard with asyncCheck

* Small fixes and optimization

* Use stew endian2 rather then system endian module

* Update nimble deps (chronos)

* Minor cosmetic/code sanity fixes

* Noise, handle Nonce max

* Noise tests, make sure to close secured conns

* More polish, improve code readability too

* More polish and testing again which test fails

* Further polishing

* Restore noise tests

* Remove useless Future[void]

* Remove useless CipherState initializer

* add a proper read wait future in second noise test

* Remove noise generic secure implementation for now

* Few fixes to run eth2 sim

* Add more debug info in noise traces

* Merge size + payload write in sendEncryptedMessage

* Revert secure interface, add outgoing property directly in newNoise

* remove sendEncrypted and receiveEncrypted

* Use openarray in chachapoly and curve25519 helpers
2020-03-17 13:30:01 +01:00
Dmitriy Ryajov 5c234f704d added 1 byte jitter test 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov dc97595aad remove irrelevant comment 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 59abd2e4cd remove unnecesary while 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 4fc84cbe81 add jitter tolerance to mplex 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 8c96de5397 reduce waiting time from 5 seconds to 500 millis in msg size test 2020-03-10 12:43:32 -06:00
Eugene Kabanov 5701d937c8
Signed variable integers fixes. (#96)
* Fix signed varints.
Add tests for signed varints.
Remove some casts to allow usage at compile time.

* Fix vsizeof() on 32bit platforms.

* Add `hint` and `zint` types for proper signed integer encoding.

* Fix varint related bugs.

* Update requirements.

* Fix interop tests because of fixed readLine.

* Add putVarint, getVarint and tests.
2020-03-06 20:19:43 +01:00
Eugene Kabanov 381630f185
Fix and refactoring of some procedures which are able to return nil as result (#97)
* Fix do not return nil as result.

* Fix mplex test to properly raise.
2020-03-04 21:45:14 +02:00
Dmitriy Ryajov ffc3b04222 read should await for data on an empty buff 2020-03-01 04:06:42 -06:00
Dmitriy Ryajov 5a00510b1f wip: increase timeout 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov eb49d4b218 no empty proto dials and add connect method 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 9efc08cb2f fix: #83 2020-02-24 08:58:02 -06:00
Dmitriy Ryajov fbcef69891 implicitelly dial pubsub if enabled 2020-02-21 09:21:06 -06:00
Giovanni Petrantoni cb156f3260 Fix public() using proper bear mulgen 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni e0e371994c Remove implicit converters 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni f039853a19 Add few converters, finish c25519 integration tests 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 99a169b65d Curve25519 integration tests (failing, something is wrong) 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 2103aa59f3 Add chachapoly proper test 2020-02-18 09:35:14 -06:00
Dmitriy Ryajov bf70428316 revert tests order back 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 9023bf786d remove sleeps 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov acdaeb8f5d working out synchronization issues 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f6c4d2130a don't forget to await for switch to close 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 934c858542 increase timeout to allow floodsub to finish 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 681d324a10 increase timeouts to acomodate for CI runs 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 30a496de5e add TODO to explain why half closed tests failed 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 7f8eb0272e cleanup and fix tests 2020-02-16 11:31:35 -06:00
Giovanni Petrantoni ea0d167285 Fix max size write mplex test 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 20f65e0dc8 Use better async pattern in test (Notice this causes a issue to show up) 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 0d0a6e9cfb No destructors on future so gotta cancel those manually 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 6c0839db01 Revert "Remove unused module"
This reverts commit 809a7af37e888d1faf8b0be83819003d570b9212.
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 76dff5eb5b Remove unused module 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 1140f7b884 Fix write limits test by using timeout, also a proposal to use this everywhere 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 120ba0d528 Add a safety check on message size limit when pushing new data in mplex 2020-02-12 11:57:38 -05:00
Eugene Kabanov 540e79a430
Add peer lifetime feature for PeerInfo. (#77)
* Add peer lifetime feature for PeerInfo.
Refactor peerinfo to use openarrays instead of sequences.
Fix tests and examples to use arrays instead of sequences.

* Add access to lifetime Future[T] itself.
2020-02-11 12:53:39 -05:00
Giovanni Petrantoni 23712ecf3b
Lazy channels (#78)
* Implemented lazy stream opening for mplex connections

* Properly fix newStream usage

* Make lazy channel open optional

* Add Lazy channel test

* Cleanup mplex test

* Move lazyness properly into LPChannel

* Connection writeLp back to proc
2020-02-11 12:30:36 -05:00
Eugene Kabanov d902127595 Fix LibP2P varint implementation to follow latest specification. (#65)
* Fix LibP2P varint implementation to decode only minimal valus.
2020-01-28 16:29:42 +01:00
Dmitriy Ryajov d42833947a fix gossipsub mesh test 2020-01-09 21:59:27 -06:00
Dmitriy Ryajov bb0430e62b remove unnecesary clear timers call 2020-01-09 14:46:00 -06:00
Dmitriy Ryajov d5f92663bc make tests pass 2020-01-09 12:55:21 -06:00
Dmitriy Ryajov 0fb1f1c5b8 strenghten pubsub interop testing 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 8714c66353 allow empty protocol string in dial 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 68cc57669e
Feat/pubsub validators (#58)
* feat: adding validator hooks to pubsub

* expose add/remove validators on switch

* do less unnecessary copyng
2019-12-16 23:24:03 -06:00
Yuriy Glukhov e0aae6d8ac Moved base32, 58 and 64 to stew 2019-12-13 16:00:39 +02:00
Dmitriy Ryajov 293a219dbe
Cleanup (#55)
* fix: don't allow replacing pubkey

* fix: several small improvements

* removing pubkey setter

* improove error handling

* remove the use of Option[T] if not needed

* don't use optional

* fix-ci: temporarily pin p2pd to a working tag

* fix example to comply with latest changes

* bumping p2pd again to a higher version
2019-12-10 14:50:35 -06:00
Zahary Karadjov 454f658ba8
Fixes and tweaks related to the beacon node integration
* Bugfix: Dialing an already connected peer may lead to crash

* Introduced a standard_setup module allowing to instantiate
  the `Switch` object in an easier manner.

* Added `Switch.disconnect(peer)`

* Trailing space removed (sorry about polluting the diff)
2019-12-08 23:58:43 +02:00
Dmitriy Ryajov 5f6fcc3d90
extract public and private keys fields from peerid (#44)
* extract public and private keys fields from peerid

* allow assigning a public key

* cleaned up TODOs

* make pubsub prefix a const

* public key should be an `Option`
2019-12-07 10:36:39 -06:00
Dmitriy Ryajov e623e70e7b
PubSub (Gossip & Flood) Implementation (#36)
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon. 

* add close event

* wip: gossipsub

* splitting rpc message

* making message handling more consistent

* initial gossipsub implementation

* feat: nim 1.0 cleanup

* wip: gossipsub protobuf

* adding encoding/decoding of gossipsub messages

* add disconnect handler

* add proper gossipsub msg handling

* misc: cleanup for nim 1.0

* splitting floodsub and gossipsub tests

* feat: add mesh rebalansing

* test pubsub

* add mesh rebalansing tests

* testing mesh maintenance

* finishing mcache implementatin

* wip: commenting out broken tests

* wip: don't run heartbeat for now

* switchout debug for trace logging

* testing gossip peer selection algorithm

* test stream piping

* more work around message amplification

* get the peerid from message

* use timed cache as backing store

* allow setting timeout in constructor

* several changes to improve performance

* more through testing of msg amplification

* prevent gc issues

* allow piping to self and prevent deadlocks

* improove floodsub

* allow running hook on cache eviction

* prevent race conditions

* prevent race conditions and improove tests

* use hashes as cache keys

* removing useless file

* don't create a new seq

* re-enable pubsub tests

* fix imports

* reduce number of runs to speed up tests

* break out control message processing

* normalize sleeps between steps

* implement proper transport filtering

* initial interop testing

* clean up floodsub publish logic

* allow dialing without a protocol

* adding multiple reads/writes

* use protobuf varint in mplex

* don't loose conn's peerInfo

* initial interop pubsub tests

* don't duplicate connections/peers

* bring back interop tests

* wip: interop

* re-enable interop and daemon tests

* add multiple read write tests from handlers

* don't cleanup channel prematurely

* use correct channel to send/receive msgs

* adjust tests with latest changes

* include interop tests

* remove temp logging output

* fix ci

* use correct public key serialization

* additional tests for pubsub interop
2019-12-05 20:16:18 -06:00
Dmitriy Ryajov 903e79ede1
Feat/conn cleanup (#41)
Backporting proper connection cleanup from #36 to align with latest chronos changes.

* add close event

* use proper varint encoding

* add proper channel cleanup in mplex

* add connection cleanup in secio

* tidy up

* add dollar operator

* fix tests

* don't close connections prematurely

* handle closing streams properly

* misc

* implement address filtering logic

* adding pipe tests

* don't use gcsafe if not needed

* misc

* proper connection cleanup and stream muxing

* re-enable pubsub tests
2019-12-03 22:44:54 -06:00
Dmitriy Ryajov 1df16bdbce set log level to trace - not enabled by default 2019-12-02 18:43:21 -06:00
cheatfate 80e128b0d9
Tests are now testing with --threads:on.
Bump version to 0.0.2.
2019-11-06 13:01:16 +02:00
cheatfate 833e8c9406
Fix overflow problem.
Fix compilation warnings.
2019-10-29 20:51:48 +02:00
Dmitriy Ryajov 7b66198566 use init instead of toMultiAddr 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9c3b73d40b fix: floodsub with multiple nodes 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3194e3ac5b fixes after building with 0.20.2 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a1c760127b fix floodsub tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f9f75254af readding commented out tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b281f46ee5 add tests for self trigger 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 468cddeb45 style/cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3b9d34116d decrease floodsub traffic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 60b5b2dc2e add pubsub tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ed372b5fab switch tests cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 34d1a641de cleanup/test pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c7606ce2eb readMsg should never return an empty buffer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f7e5c8ee30 bring back commented tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f3afe0a6ce feat: allow ussage of 0 addrs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5b3f93ba1c feat: allow multiple handlers per topic in pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e57e10703b test: re-enable identify 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 393a9bb4e5 remove debug echos 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 011df568b7 finished integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9bb892de69 wip: integrating and testing secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 27e4516796 fix breaking test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bcaed7c6b9 refactor: identify requires Optional[T] keys 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f47d9196b1 test finishes correctly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c1983e1685 commented out outdated tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1fe2a391ae fix: temprorarily disable broken tests in mxplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ef89e98479 renaming channel 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 88079a2864 incomming connection handling flow and tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8920cd7d60 misc: pubsub/floodsub and logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 177eb71ffa wip: floodsub initial implementation 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 74d2aea63d adding chronicles topics 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0347a8ef1d test connections properly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2febd47e0d don't complete futures twise 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 435c69633f adding chronicles logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 93a9fd203a fix: multistream tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e31966b6f8 fix: switch, with identify and mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 997745b7e7 add multiple read/write streams test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e53c87e197 reworked to make msg reading sequential 2019-10-11 08:15:24 +09:00