Commit Graph

511 Commits

Author SHA1 Message Date
Dmitriy Ryajov a52763cc6d
fix publishing (#250)
* use var semantics to optimize table access

* wip... lvalues don't work properly sadly...

* big publish refactor, replenish and balance

* fix internal tests

* use g.peers for fanout (todo: don't include flood peers)

* exclude non gossip from fanout

* internal test fixes

* fix flood tests

* fix test's trypublish

* test interop fixes

* make sure to not remove peers from gossip table

* restore old replenishFanout

* cleanups

* Cleanup resources (#246)

* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises

* restore trace vs debug in gossip

* improve fanout replenish behavior further

* cleanup stale peers more eaguerly

* synchronize connection cleanup and small refactor

* close client first and call parent second

* disconnect failed peers on publish

* check for publish result

* fix tests

* fix tests

* always call close

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-07 18:33:05 -06:00
Eugene Kabanov 775cab414a
Remove SHA1 from crypto and crypto tests. (#251)
* Remove SHA1 from crypto and crypto tests.

* Simplify RSA comparison procedure.
Refactor some procedures in crypto.nim.
2020-07-07 15:48:15 +02:00
Jacek Sieka d522537b19
reuse single RNG instance for all crypto key generation (#249)
* reuse single RNG instance for all crypto key generation

* use foolproof rng

* initRng -> newRng (because it's ref)

* fix test

* imports/exports, chat fix

* fix rsa

* imports and exports

* work around threadvar issue

* fixup

* mac workaround test
2020-07-07 13:14:11 +02:00
Giovanni Petrantoni ec00c7fc50
Peer resultification and defect only (#245)
* Peer resultification and defect only

* Fixing some tests

* test fixes

* Rename peer into peerid

* better result error message in identify

* further merge fixes
2020-07-01 08:25:09 +02:00
Dmitriy Ryajov c788a6a3c0
Cleanup resources (#246)
* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises
2020-06-29 09:15:31 -06:00
Jacek Sieka aa6756dfe0
allow message id provider to be specified (#243)
* don't send public key in message when not signing (information leak)
* don't run rebalance if there are peers in gossip (see #242)
* don't crash randomly on bad peer id from remote
2020-06-28 09:56:38 -06:00
Dmitriy Ryajov 902880ef1f
consolidate reading in lpstream (#241)
* consolidate reading in lpstream

* remove debug echo

* throw if not enough bytes where read

* tune log level

* set eof flag

* test readExactly to fail on not enough bytes
2020-06-27 11:33:34 -06:00
Dmitriy Ryajov 7a95f1844b
Concurrent dials (#238)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* add secureconn constructor

* close in the correct order

* concurent dial lock and track in/out conns better

* make tests pass

* add todo comment

* disconect peers that open too many connections

* wip

* do connection and muxer tracking in one place

* prevent nil pointer in observers

* drop connections when peers is over max

* prevent channel leaks

* don't use closure to handle channel
2020-06-24 09:08:44 -06:00
Giovanni Petrantoni ee6e545878
multistream select make sure to not report NA (#235)
* multistream select make sure to not report NA but rather empty string if all fails

Also re-enable tests

* avoid using bad constructs, make multistream.select flow crystal clear
2020-06-22 15:38:48 -06:00
Jacek Sieka 6331b04cb4
secp: requiresInit updates (#237)
* secp: requiresInit updates

* fix
2020-06-22 19:03:15 +02:00
Giovanni Petrantoni 7852c6dd0f
Noise and eth2/nbc fixes (#226)
* Remove noise padding payload (spec removed it)

* add log scope in secure

* avoid defect array out of range in switch secure when "na"

* improve identify traces

* wip noise fixes

* noise protobuf adjustments (trying)

* add more debugging messages/traces, improve their actual contents

* re-enable ID check in noise

* bump go daemon tag version

* bump go daemon tag version

* enable noise in daemonapi

* interop testing, (both secio and noise will be tested)

* azure cache bump (p2pd)

* CI changes

- Travis: use Go 1.14
- azure-pipelines.yml: big cleanup
- Azure: bump cache keys
- build 64-bit p2pd on 32-bit Windows
- install both Mingw-w64 architectures

* noise logging fixes

* alternate testing between noise and secio

* increase timeout to avoid VM errors in CI (multistream tests)

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2020-06-20 19:56:55 +09:00
Dmitriy Ryajov 5b28e8c488
Cleanup lpstream, Connection and BufferStream (#228)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* don't use delegation in connection

* move connection out to own file

* don't breakout on reset

* make sure to call close on secured conn

* add lpstream tracing

* don't breackdown by conn id

* fix import

* remove unused lable

* reset  connection on exception

* add additional metrics for skipped messages

* check for nil in secure.close
2020-06-19 11:29:43 -06:00
Dmitriy Ryajov 5960d42c50
remove casts from (#203) 2020-06-02 20:21:11 -06:00
Dmitriy Ryajov bb8bff2195
add sparse message propagation tests to gossipsub (#202)
* add sparce tests to gossipsub

* add send hooks

* remove `all`
2020-06-02 17:53:38 -06:00
Dmitriy Ryajov 285884c20c
Close peers (#201)
* wip

* exceptions and resource cleanup

* correct peerlifetime on disconnect

* emulate defered

* remove comment
2020-06-02 11:32:42 -06:00
Dmitriy Ryajov abbb50ae34 add leaks tests 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov abf659a01a more consistent dialing proto selecting logic 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 4c191866e4 use all() for futures and track connections 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 4df151a3a3 typos 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov a4277cf39a test for openned/closed resource 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov daef00fc7b don't crash schlesi-dev 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 20c68a2018 use all() for futures and track connections 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 6112de746d remove unneeded changes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 5f704e6825 rust interop fixes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 8e48bb117d rebasing master 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 77017d1e7e inc count before closing 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 8963c2d0ed fixes after backporting 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov bfe84816ac don't reraise just yet 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 87e1cf924f don't use read 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 46daed9a38 wip 2020-06-02 09:10:27 -06:00
Giovanni Petrantoni 37b98ad45c
Secure managers are now sorted, giving priority to noise (#191)
* Secure managers are now sorted, giving priority to noise

* fix nimble test command

* Fix native tests

* fix directchat sample

* Could not write to connection - reduce verbosity

* fix interop testing

* Remove more tables

* test interop fixes

* directchat fix

* fix interop/remove some deprecation
2020-06-01 08:41:32 +02:00
Giovanni Petrantoni 6affcda937
Less exceptions more results (#188)
* Less exceptions more results

* Fix daemonapi and interop tests

* Add multibase

* wip multiaddress

* fix the build, consuming new result types

* fix standard setup

* Simplify match, rename into MaError, add more exaustive err text

* Fix the CI issues

* Fix directchat build

* daemon api fixes

* better err messages formatting

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2020-05-31 16:22:49 +02:00
Dmitriy Ryajov 7b6e1c0688
Gossipsub interop (#189)
* interop fixes

* add custom messageid provider and fix seqno

* use ECDSA for speed

* adding messageid tests

* breakout from publish loop

* addressing review comments

* remove unneded var

* dont stop broadcasting on failed peers
2020-05-27 12:33:49 -06:00
Dmitriy Ryajov 9132f16927
gossipsub fixes (#186) 2020-05-21 14:24:20 -06:00
Dmitriy Ryajov ba53c08b3c
Track incoming connections (#181)
* call write until all is written out

* wip: rework with proper half-closed

* add eof and closed handling

* wip

* close connection on chronos close

* don't use read

* make noise work again

* don't reraise just yet

* fixes after backporting

* remove on transport close cleanup

* revert back allread

* rust interop fixes

* read from stream

* inc count before closing

* rebasing master

* store incomming connections

* fix merge

* remove unneeded changes

* use internal close flag to indicate disposal
2020-05-21 11:33:48 -06:00
Dmitriy Ryajov 7900fd9f61
Half closed (#174)
* call write until all is written out

* add comments to lpchannel fields

* add an eof flag to signal which end closed

* wip: rework with proper half-closed

* add eof and closed handling

* propagate closes to piped

* call parent close

* moving bufferstream trackers out

* move writeLock to bufferstream

* move writeLock out

* remove unused call

* wip

* rebasing master

* fix mplex tests

* wip

* fix bufferstream after backport

* wip

* rename to differentiate from chronos tracker

* close connection on chronos close

* make reset request asyncCheck

* fix channel cleanup

* misc

* don't use read

* fix backports

* make noise work again

* proper exception handling

* don't reraise just yet

* add convenience templates

* dont double wrap

* use async pragma

* fixes after backporting

* muxer owns connection

* remove on transport close cleanup

* revert back allread

* adding some todos

* read from stream

* inc count before closing

* rebasing master

* rebase master

* use correct exception type

* use try/finally insted of defer

* fix compile in trace mode

* reset channels on mplex close
2020-05-19 18:14:15 -06:00
Dmitriy Ryajov 9cf1fd0216
remove generic constructor and expose serverflags (#176)
* remove generic constructor and expose serverflags

* fix transport constructor

* fix merge issues
2020-05-18 13:04:05 -06:00
Dmitriy Ryajov 773b738c12
don't track Connection, track StreamTransport (#177)
* don't track Connection, track StreamTransport

* make tests more deterministic
2020-05-18 11:05:34 -06:00
Dmitriy Ryajov 1819502fb5
Cleanup - tests and logging (#178)
* make async for proper exception handling

* tryAndWarn msg messes up Exception msg

* misc: comment out tracker dumps

* cleanup mplex tests

* more informative errors

* give CI time to run

* revert change, bacause it causes races
2020-05-18 07:49:49 -06:00
Giovanni Petrantoni 7dcb807f64
Crypto utilities resultification (#150) 2020-05-18 07:25:55 +02:00
Dmitriy Ryajov 167f42ed45
Remove read (#171)
* use readExactly

* remove `read`

* remove read

* no more `read`
2020-05-14 22:02:05 -06:00
Jacek Sieka 69abf5097d
handle a few exceptions (#170)
* handle a few exceptions

Some of these are maybe too aggressive, but in return, they'll log
their exception - more refactoring needed to sort this out - right now
we get crashes on unhandled exceptions of unknown origin

* during connection setup
* while closing channels
* while processing pubsubs

* catch exceptions that are raised and don't try to catch exceptions that are not raised

* propagate cancellederror

* one more

* more

* more

* make interop tests less fragile

* Raise expiration time in gossipsub fanout test for slow CI

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-05-14 21:56:56 -06:00
Giovanni Petrantoni 100d6ef595 Raise expiration time in gossipsub fanout test for slow CI 2020-05-15 11:01:33 +09:00
Jacek Sieka 3053f03814 fix varint issues
* fixes #111
2020-05-11 09:12:23 -06:00
Jacek Sieka ccd019b328
use stream directly in chronosstream (#163)
* use stream directly in chronosstream

for now, chronos.AsyncStream is not used to provide any features on top
of chronos.Stream, so in order to simplify the code, chronosstream can
be used directly.

In particular, the exception handling is broken in the current
chronosstream - opening and closing the stream is simplified this way as
well.

A future implementation that actually takes advantage of the AsyncStream
features would wrap AsyncStream instead as a separate lpstream
implementation, leaving this one as-is.

* work around chronos exception type issue
2020-05-08 22:10:06 +02:00
Giovanni Petrantoni c889224012 Add PubSub observer+ hooks (they can modify as well) 2020-05-08 13:31:52 -06:00
Jacek Sieka 1efada474c
remove readLoop in secure protocols (#162)
* remove readLoop in secure protocols, fix security issues

* fix Defect on remote sending 0-byte noise/secio message
* remove msglen from `write` (unused)
* simplify SecureConn data flow
* document some control-flow issues

* unify exception behaviour across noise and secio

* secio would not raise on mac/decryption errors

* fix compile error
2020-05-07 14:37:46 -06:00
Jacek Sieka 330da51819
removals (#159)
* remove unused stream methods
* reimplement some of them with proc's
* remove broken tests
* Error->Defect for defect
* warning fixes
2020-05-06 18:31:47 +02:00
Dmitriy Ryajov 6da4d2af48
Pubsub signatures flags (#161)
* add verify signature flag

* add sign flag to enable/disable msg signing

* moving internal tests out to their own file

* cleanup nimble file

* remove unneeded tests

* move pubsub tests out

* fix tests
2020-05-06 11:26:08 +02:00
cheatfate 290ba712e9
Fix MultiAddress.protoAddress() bug for fixed arrays.
Add tests for it.
2020-04-28 14:43:44 +03:00
cheatfate 917b5f5c84 Add MultiAddress.init(integer) for tcp,udp,dccp,sctp protocols.
Add tests for it.
2020-04-23 08:10:17 -06:00
Giovanni Petrantoni 1c4d72f5e3
Use Result construct in minasn1 (#144) 2020-04-23 14:10:20 +02:00
Giovanni Petrantoni 4c6a123d31
Add chronos trackers and used them to sanitize resource disposal (#131)
* Add chronos trackers and used them to sanitize resource disposal

* Chronos trackers for transport tests wip

* No more chronos leaks in testtransport

* Make tcp transport and test more robust when closing

* Test async leaking tracking wip

* Fix a regression in wire connect

* Add chronos trackers to more tests and sanitize resource closure

* Wip fixing floodsub tests

* Floodsub wip

* Made floodsub basically deterministic, hit a nim bug with captures tho

* Wrap up floodsub tests refactor

* Wrapping up

* Add allFuturesThrowing utility

* Fix missing allFuturesThrowing in noise tests!

* Make tests green

* attempt fixing gossipsub failing cases

* Make sure to check also fanout in waitSub

* More verbose traces

* Gossipsub test improvments

* Refactor TcpTransport remove asyncCheck

* Add Connection trackers

* Add stricter connection tracking, wip mplex fix

* More asynccheck removal, in order to avoid connection leaks

* bump chronicles requirement

* Enable tracker dump to check CI output

* Wait for more futures in testmplex

* Remove tracker dump messages

* add tryAndWarn utility, fix mplex issue with go interop

* All allFuturesThrowing to directchat too

* make sure to cleanup on transport close
2020-04-21 10:24:42 +09:00
Jacek Sieka e8b33c64fa
secp: use upstream secp convenience api (#141)
* secp: use upstream secp convenience api
2020-04-17 12:51:13 +02:00
Dmitriy Ryajov 4ee1b4a66d connection closing tests 2020-04-07 12:16:59 -06:00
Giovanni Petrantoni e39bf0a4cf
Mplex: Add the ability to send any size payload (#123)
* Mplex: Add the ability to send any size payload

* Ensure size of coder header
2020-04-03 17:26:46 +02:00
Ștefan Talpalaru 7f8090b166
SecioConn.readMessage(): fix the exception types 2020-04-02 19:07:14 +02:00
Dmitriy Ryajov 5285f0d091
Fix/misc (#116)
* only check for payload size

* only subscribe if connection succeeded

* fix failing test

* check that the strem is active before openning

* msg type should not be > than 0x7

* fix tests

* check max against enum val
2020-03-29 08:28:48 -06:00
Dmitriy Ryajov 6bb4e91a39 fix tests 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov a92d616a3e fix tests 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov 3effb95f10 close underlying bufferstream in lpchannel 2020-03-28 09:29:43 -06:00
Giovanni Petrantoni c0a10ecea7
Disable new fragmentation tests (#119)
Merging for now to fix tests (virtually)
Could not repro locally yet
2020-03-27 10:47:30 +09:00
Giovanni Petrantoni 56392913f9
Allow to specify a read size in readLp (#117)
* Allow to specify a read size in readLp

Also fix secio fragments + test

* Revert readLp changes, use just `read` in test
2020-03-26 15:06:47 +09:00
Giovanni Petrantoni 1550bea1cc Support arbitrary stream write sizes in noise 2020-03-25 16:10:11 +09:00
cheatfate a2acdd7933
Disable unreliable provide cid test. 2020-03-24 10:05:11 +02:00
Giovanni Petrantoni c02fca25f8
Noise (#90)
* Start ChaCha20Poly1305 integration (BearSSL)

* Add Curve25519 (BearSSL) required operations for noise

* Fix curve mulgen iterate/derive

* Fix misleading header

* Add chachapoly proper test

* Curve25519 integration tests (failing, something is wrong)

* Add few converters, finish c25519 integration tests

* Remove implicit converters

* removed internal globals

* Start noise implementation

* Fix public() using proper bear mulgen

* Noise protocol WIP

* Noise progress

* Add a quick nim version of HKDF

* Converted hkdf to iterator, useful for noise

* Noise protocol implementation progress

* Noise progress

* XX handshake almost there

* noise progress

* Noise, testing handshake with test vectors

* Noise handshake progress, still wrong somewhere!

* Noise handshake success!

* Full verified noise XX handshake completed

* Fix and rewrite test to be similar to switch one

* Start with connection upgrade

* Switch chachapoly to CT implementations

* Improve HKDF implementation

* Use a type insted of tuple for HandshakeResult

* Remove unnecessary Let

* More cosmetic fixes

* Properly check randomBytes result

* Fix chachapoly signature

* Noise full circle (altho dispatcher is nil cursed)

* Allow nil aads in chachapoly routines

* Noise implementation up to running full test

* Use bearssl HKDF as well

* Directly use bearssl rng for curve25519 keys

* Add a (disabled/no CI) noise interop test server

* WIP on fixing interop issues

* More fixes in noise implementation for interop

* bump chronos requirement (nimble)

* Add a chachapoly test for very small size payloads

* Noise, more tracing

* Add 2 properly working noise tests

* Fix payload packing, following the spec properly (and not go version but
rather rust)

* Sanity, replace discard with asyncCheck

* Small fixes and optimization

* Use stew endian2 rather then system endian module

* Update nimble deps (chronos)

* Minor cosmetic/code sanity fixes

* Noise, handle Nonce max

* Noise tests, make sure to close secured conns

* More polish, improve code readability too

* More polish and testing again which test fails

* Further polishing

* Restore noise tests

* Remove useless Future[void]

* Remove useless CipherState initializer

* add a proper read wait future in second noise test

* Remove noise generic secure implementation for now

* Few fixes to run eth2 sim

* Add more debug info in noise traces

* Merge size + payload write in sendEncryptedMessage

* Revert secure interface, add outgoing property directly in newNoise

* remove sendEncrypted and receiveEncrypted

* Use openarray in chachapoly and curve25519 helpers
2020-03-17 13:30:01 +01:00
Dmitriy Ryajov 5c234f704d added 1 byte jitter test 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov dc97595aad remove irrelevant comment 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 59abd2e4cd remove unnecesary while 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 4fc84cbe81 add jitter tolerance to mplex 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 8c96de5397 reduce waiting time from 5 seconds to 500 millis in msg size test 2020-03-10 12:43:32 -06:00
Eugene Kabanov 5701d937c8
Signed variable integers fixes. (#96)
* Fix signed varints.
Add tests for signed varints.
Remove some casts to allow usage at compile time.

* Fix vsizeof() on 32bit platforms.

* Add `hint` and `zint` types for proper signed integer encoding.

* Fix varint related bugs.

* Update requirements.

* Fix interop tests because of fixed readLine.

* Add putVarint, getVarint and tests.
2020-03-06 20:19:43 +01:00
Eugene Kabanov 381630f185
Fix and refactoring of some procedures which are able to return nil as result (#97)
* Fix do not return nil as result.

* Fix mplex test to properly raise.
2020-03-04 21:45:14 +02:00
Dmitriy Ryajov ffc3b04222 read should await for data on an empty buff 2020-03-01 04:06:42 -06:00
Dmitriy Ryajov 5a00510b1f wip: increase timeout 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov eb49d4b218 no empty proto dials and add connect method 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 9efc08cb2f fix: #83 2020-02-24 08:58:02 -06:00
Dmitriy Ryajov fbcef69891 implicitelly dial pubsub if enabled 2020-02-21 09:21:06 -06:00
Giovanni Petrantoni cb156f3260 Fix public() using proper bear mulgen 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni e0e371994c Remove implicit converters 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni f039853a19 Add few converters, finish c25519 integration tests 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 99a169b65d Curve25519 integration tests (failing, something is wrong) 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 2103aa59f3 Add chachapoly proper test 2020-02-18 09:35:14 -06:00
Dmitriy Ryajov bf70428316 revert tests order back 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 9023bf786d remove sleeps 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov acdaeb8f5d working out synchronization issues 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f6c4d2130a don't forget to await for switch to close 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 934c858542 increase timeout to allow floodsub to finish 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 681d324a10 increase timeouts to acomodate for CI runs 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 30a496de5e add TODO to explain why half closed tests failed 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 7f8eb0272e cleanup and fix tests 2020-02-16 11:31:35 -06:00
Giovanni Petrantoni ea0d167285 Fix max size write mplex test 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 20f65e0dc8 Use better async pattern in test (Notice this causes a issue to show up) 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 0d0a6e9cfb No destructors on future so gotta cancel those manually 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 6c0839db01 Revert "Remove unused module"
This reverts commit 809a7af37e.
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 76dff5eb5b Remove unused module 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 1140f7b884 Fix write limits test by using timeout, also a proposal to use this everywhere 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 120ba0d528 Add a safety check on message size limit when pushing new data in mplex 2020-02-12 11:57:38 -05:00
Eugene Kabanov 540e79a430
Add peer lifetime feature for PeerInfo. (#77)
* Add peer lifetime feature for PeerInfo.
Refactor peerinfo to use openarrays instead of sequences.
Fix tests and examples to use arrays instead of sequences.

* Add access to lifetime Future[T] itself.
2020-02-11 12:53:39 -05:00
Giovanni Petrantoni 23712ecf3b
Lazy channels (#78)
* Implemented lazy stream opening for mplex connections

* Properly fix newStream usage

* Make lazy channel open optional

* Add Lazy channel test

* Cleanup mplex test

* Move lazyness properly into LPChannel

* Connection writeLp back to proc
2020-02-11 12:30:36 -05:00
Eugene Kabanov d902127595 Fix LibP2P varint implementation to follow latest specification. (#65)
* Fix LibP2P varint implementation to decode only minimal valus.
2020-01-28 16:29:42 +01:00
Dmitriy Ryajov d42833947a fix gossipsub mesh test 2020-01-09 21:59:27 -06:00
Dmitriy Ryajov bb0430e62b remove unnecesary clear timers call 2020-01-09 14:46:00 -06:00
Dmitriy Ryajov d5f92663bc make tests pass 2020-01-09 12:55:21 -06:00
Dmitriy Ryajov 0fb1f1c5b8 strenghten pubsub interop testing 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 8714c66353 allow empty protocol string in dial 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 68cc57669e
Feat/pubsub validators (#58)
* feat: adding validator hooks to pubsub

* expose add/remove validators on switch

* do less unnecessary copyng
2019-12-16 23:24:03 -06:00
Yuriy Glukhov e0aae6d8ac Moved base32, 58 and 64 to stew 2019-12-13 16:00:39 +02:00
Dmitriy Ryajov 293a219dbe
Cleanup (#55)
* fix: don't allow replacing pubkey

* fix: several small improvements

* removing pubkey setter

* improove error handling

* remove the use of Option[T] if not needed

* don't use optional

* fix-ci: temporarily pin p2pd to a working tag

* fix example to comply with latest changes

* bumping p2pd again to a higher version
2019-12-10 14:50:35 -06:00
Zahary Karadjov 454f658ba8
Fixes and tweaks related to the beacon node integration
* Bugfix: Dialing an already connected peer may lead to crash

* Introduced a standard_setup module allowing to instantiate
  the `Switch` object in an easier manner.

* Added `Switch.disconnect(peer)`

* Trailing space removed (sorry about polluting the diff)
2019-12-08 23:58:43 +02:00
Dmitriy Ryajov 5f6fcc3d90
extract public and private keys fields from peerid (#44)
* extract public and private keys fields from peerid

* allow assigning a public key

* cleaned up TODOs

* make pubsub prefix a const

* public key should be an `Option`
2019-12-07 10:36:39 -06:00
Dmitriy Ryajov e623e70e7b
PubSub (Gossip & Flood) Implementation (#36)
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon. 

* add close event

* wip: gossipsub

* splitting rpc message

* making message handling more consistent

* initial gossipsub implementation

* feat: nim 1.0 cleanup

* wip: gossipsub protobuf

* adding encoding/decoding of gossipsub messages

* add disconnect handler

* add proper gossipsub msg handling

* misc: cleanup for nim 1.0

* splitting floodsub and gossipsub tests

* feat: add mesh rebalansing

* test pubsub

* add mesh rebalansing tests

* testing mesh maintenance

* finishing mcache implementatin

* wip: commenting out broken tests

* wip: don't run heartbeat for now

* switchout debug for trace logging

* testing gossip peer selection algorithm

* test stream piping

* more work around message amplification

* get the peerid from message

* use timed cache as backing store

* allow setting timeout in constructor

* several changes to improve performance

* more through testing of msg amplification

* prevent gc issues

* allow piping to self and prevent deadlocks

* improove floodsub

* allow running hook on cache eviction

* prevent race conditions

* prevent race conditions and improove tests

* use hashes as cache keys

* removing useless file

* don't create a new seq

* re-enable pubsub tests

* fix imports

* reduce number of runs to speed up tests

* break out control message processing

* normalize sleeps between steps

* implement proper transport filtering

* initial interop testing

* clean up floodsub publish logic

* allow dialing without a protocol

* adding multiple reads/writes

* use protobuf varint in mplex

* don't loose conn's peerInfo

* initial interop pubsub tests

* don't duplicate connections/peers

* bring back interop tests

* wip: interop

* re-enable interop and daemon tests

* add multiple read write tests from handlers

* don't cleanup channel prematurely

* use correct channel to send/receive msgs

* adjust tests with latest changes

* include interop tests

* remove temp logging output

* fix ci

* use correct public key serialization

* additional tests for pubsub interop
2019-12-05 20:16:18 -06:00
Dmitriy Ryajov 903e79ede1
Feat/conn cleanup (#41)
Backporting proper connection cleanup from #36 to align with latest chronos changes.

* add close event

* use proper varint encoding

* add proper channel cleanup in mplex

* add connection cleanup in secio

* tidy up

* add dollar operator

* fix tests

* don't close connections prematurely

* handle closing streams properly

* misc

* implement address filtering logic

* adding pipe tests

* don't use gcsafe if not needed

* misc

* proper connection cleanup and stream muxing

* re-enable pubsub tests
2019-12-03 22:44:54 -06:00
Dmitriy Ryajov 1df16bdbce set log level to trace - not enabled by default 2019-12-02 18:43:21 -06:00
cheatfate 80e128b0d9
Tests are now testing with --threads:on.
Bump version to 0.0.2.
2019-11-06 13:01:16 +02:00
cheatfate 833e8c9406
Fix overflow problem.
Fix compilation warnings.
2019-10-29 20:51:48 +02:00
Dmitriy Ryajov 7b66198566 use init instead of toMultiAddr 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9c3b73d40b fix: floodsub with multiple nodes 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3194e3ac5b fixes after building with 0.20.2 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a1c760127b fix floodsub tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f9f75254af readding commented out tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b281f46ee5 add tests for self trigger 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 468cddeb45 style/cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3b9d34116d decrease floodsub traffic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 60b5b2dc2e add pubsub tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ed372b5fab switch tests cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 34d1a641de cleanup/test pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c7606ce2eb readMsg should never return an empty buffer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f7e5c8ee30 bring back commented tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f3afe0a6ce feat: allow ussage of 0 addrs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5b3f93ba1c feat: allow multiple handlers per topic in pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e57e10703b test: re-enable identify 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 393a9bb4e5 remove debug echos 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 011df568b7 finished integrating secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9bb892de69 wip: integrating and testing secio 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 27e4516796 fix breaking test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bcaed7c6b9 refactor: identify requires Optional[T] keys 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f47d9196b1 test finishes correctly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c1983e1685 commented out outdated tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1fe2a391ae fix: temprorarily disable broken tests in mxplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ef89e98479 renaming channel 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 88079a2864 incomming connection handling flow and tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8920cd7d60 misc: pubsub/floodsub and logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 177eb71ffa wip: floodsub initial implementation 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 74d2aea63d adding chronicles topics 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0347a8ef1d test connections properly 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2febd47e0d don't complete futures twise 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 435c69633f adding chronicles logging 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 93a9fd203a fix: multistream tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e31966b6f8 fix: switch, with identify and mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 997745b7e7 add multiple read/write streams test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e53c87e197 reworked to make msg reading sequential 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f761a7050e read/write order test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ba843f4091 wip: interop testing 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8cd1b71781 cosmetic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 13ca6d253d fix: multistream tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0f52a6e798 fix: switch and tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6f8de062bb fix: multistream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5237fc8b12 test multiple streams with mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 746987156d reworked switch 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0b784c5b58 don't pass stream handler through contructor 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9889bd9cbf return proto string from select, instead of bool 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cc595f7947 allow selecting one of many protos in identify 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a7e5fde6f7 check identify request against known pubkey 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ec351cc2b0 misc: cleanup mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6ce2782e5c add missing test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9b20dbc7ae add mplex tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9b485b3082 feat: half closed channels 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6058a3fc69 split mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c2ce55a94a finish basic tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 36917c5adc disable switch tests temporarily 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1611d2b1c0 readding missed tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a0aaa29739 wip: mplex and mplex tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f4a9b9cba9 test: rework remove generic protocol constructor 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0c0184ca63 fix: tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b4dced6186 fix: use custom constructor 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 96cd7bcf50 wip: mplex tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 12702e2cd2 adding bufferstream tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 7f0627378c misc 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov fceea14aa5 adding bufferred stream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cbf0f4f186 move streams to own dir 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 85b0b5d1da don't do pointless cast 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ef7a0b41e4 add switch tests to native 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov db1aad3df0 adding switch e2e test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 79129ea6d1 `handles` should be called on concrete transport 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3df92e0d67 wip: switch 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 022a248ca3 reworked protocol 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 22dd8c0f6b add native tests to testnative 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e2b04fc30d add gcsafe pragma 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0f42be495b remove dead code 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ca4ccbe37f don't use method as handler in protocol base 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5eced0abee rename multistream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov aa2dfababd wip: identify 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b85d616a13 call the handler with the protocol context 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 8d4de6b587 remove reduntant test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 002328f7e5 don't use conflicting ips 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3d7f657ce8 rename readerwriter to stream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov d23398f498 nimpretty 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ebab744106 end to end tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 97788f1c49 end to end tests for multistream 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 77aa11f0f3 removing bins 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 573a568946 fix tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 64e92eb9da adding `na` test/functionality 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 4dcb542ae8 add `ls` test 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e13f42f9bb wip: initial multistream handler implementation 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 307c76e139 feat: implemented select 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5d07145a0a feat: added stream abstraction and chronos wrapper 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5f8e633c0f feat: use error pragma to signal abstract methods 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a91c15c47a feat: adding dial/listen test with transports on both ends 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c15a9bdd67 feat: adding dial tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c8546583e6 feat: tcp transport listen 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 47106a6a7d wip: adding test 2019-10-11 08:15:24 +09:00
cheatfate 7138f7e94d
Add more primitives for SecIO.
Fix SIGSEGV inside of rsa.nim and ecnist.nim.
2019-09-11 19:03:39 +03:00
cheatfate 968e635746
Add secp256k1 curve support and tests for it. 2019-09-02 21:55:42 +03:00
cheatfate 2ab6b2c3ca
Fix #17.
Add go-libp2p-daemon logging to file.
Fix mistypes in tests.
Fix issues with connection to dedicated go-libp2p2-daemon.
2019-08-22 00:42:44 +03:00
cheatfate dfd824bd03
Fix #22.
Fix wire.initTAddress(MultiAddress).
Add Windows support to daemonapi.
Add CI tests for daemonapi on Windows.
Fix Linux CI tests to install latest daemon.
2019-08-01 08:56:59 +03:00
cheatfate d685147c52
Move ipnet/iface functionality to nim-chronos.
Adjust tests.
2019-04-15 11:16:54 +03:00
cheatfate ab8b87be9a
Fix deprecation warnings. 2019-03-31 01:32:04 +02:00
cheatfate f8dc3abe36
Add MultiAddress pattern matching procedures (go-multiaddr-fmt) with tests.
Add some comments.
2019-03-20 11:41:37 +02:00
cheatfate 518d33e26c
Add cross-platform getInterfaces() and test. 2019-03-18 10:42:30 +02:00
cheatfate 9ddc47fa56
IpMask, IpNet and tests initial commit. 2019-03-18 01:18:10 +02:00
cheatfate abde17e8c2
Prepare for Windows tests. 2019-03-16 00:24:33 +02:00
cheatfate dc2d0538ff
Fix import bugs.
Add PublicKey and Signature types to daemonapi.
Add more tests for crypto.nim
2019-03-06 13:23:12 +02:00
cheatfate b185ad79f8
Add PeerID test vectors. 2019-03-06 03:48:22 +02:00
cheatfate 39129d0ec1
Add base64 and tests for it.
Adopt multibase to be able to use base64.
Fix base32 typos.
2019-03-06 02:36:09 +02:00
cheatfate 215c6c4257
Add SHA-1 support to multihash.nim, enabled tests for it. 2019-03-05 11:49:59 +02:00
cheatfate 77f34af737
Fix testdaemon.nim. 2019-03-04 20:51:10 +02:00
cheatfate b235c84bb2
Add key interface with test vectors. 2019-03-02 21:19:41 +02:00
cheatfate 14686714c0
Add raw init functions for ecnist.nim
Add ECDHE helpers for ecnist.nim
Add test vectors for ECDHE.
2019-03-01 08:34:52 +02:00
cheatfate 2a2e378ced
Fix checkScalar in ecnist.
Add ed25519 and tests.
Add some comments.
2019-02-28 14:17:39 +02:00
cheatfate 39fb9ed2de Fix fail verification tests. 2019-02-25 11:06:50 +02:00
cheatfate 887795bbd2 Final RSA changes.
Add RSA tests and test vectors.
2019-02-24 02:07:13 +02:00
cheatfate 1aaf4797c9 Finish EC NIST curves implementation.
Add EC NIST curves tests and test vectors.
2019-02-22 13:32:15 +02:00
cheatfate 64d9895f25 Fix tests rename asyncdispatch -> chronos. 2019-02-14 03:32:29 +02:00
cheatfate 7593d4d970 Fix unix path multiaddress serialization/deserialization.
Add tests for unix path multiaddress.
Add wire.nim.
Fix latest breaking changes from go-libp2p-daemon.
2019-01-09 19:12:15 +02:00
cheatfate 7e2d3e213f Add isWire() procedure for MultiAddress. 2018-12-29 14:44:27 +02:00
cheatfate bf2737525d Fix cid with proper CIDv1 code.
Fix daemonapi to use proper Cid type.
Make daemonapi cid test more complex.
2018-12-16 15:51:12 +02:00
cheatfate bd7c4d646f Initial CID implementation. 2018-12-16 04:55:02 +02:00
cheatfate 9c2b8591d4 Got fix for #8 in go-libp2p-pubsub.
Enable full test.
2018-12-14 11:00:49 +02:00
cheatfate 0b807e7ee5 Workaround for bug https://github.com/libp2p/go-libp2p-pubsub/issues/130. 2018-12-13 11:17:02 +02:00
cheatfate 2ca85ab950 Show logs if something goes wrong. 2018-12-12 16:20:26 +02:00
cheatfate 0eb649c990 Add logging `go-libp2p-daemon`.
Comment FloodSub test.
2018-12-12 16:08:55 +02:00
cheatfate 7d60e22782 MultiBase initial implementation with tests. 2018-12-12 03:52:42 +02:00
cheatfate fc6902dda2 Add PeerInfo string representation procedure.
Use only IPv4 for pubsub tests.
2018-12-11 04:17:36 +02:00
cheatfate 01a268a440 Attempt to fix tests. 2018-12-10 22:55:06 +02:00
cheatfate 337c6c932c Fix go-libp2p-daemon shutdown. 2018-12-10 12:38:12 +02:00
cheatfate 7e5f52afff Fixes for pubsub methods and full tests for both methods. 2018-12-09 18:44:20 +02:00
cheatfate bdcfa1f189 Add multihash test. 2018-12-05 03:01:00 +02:00
cheatfate 53fc4cce41 Fix MultiAddress Onion and P2P address validation and encoding/decoding.
Enabled Onion/P2P tests.
Add nimcrypto dependency to nimble file.
2018-12-05 02:30:50 +02:00
cheatfate 3abf7df73d Fix base32 encoding and added more tests. 2018-12-03 16:59:40 +02:00
cheatfate 04b4d8f688 Add base32 encoding/decoding procedures and tests. 2018-12-03 15:07:14 +02:00
cheatfate 98aa3ed281 Add VBuffer implementation.
Add MultiAddress implementation and tests.
Adapt chat to use /p2p-circuit
2018-11-27 14:16:04 +02:00
cheatfate cd190e62c2 Add base58 encoding and tests. 2018-11-24 00:06:06 +02:00
cheatfate 78cafd9156 Fix test compilation error. 2018-11-23 12:20:53 +02:00
cheatfate 9edc823b6c Some fixes to chat, and one more test for bad cid provide. 2018-11-22 17:35:44 +02:00
cheatfate 3b554d0f51 Add LibP2P varint and tests.
Generalize Varint.
2018-11-20 23:39:35 +02:00
cheatfate 152a93e25b Fix daemon test close api. 2018-11-19 22:53:20 +02:00
cheatfate f49d67bc0d Add one more test for self-connect-open-accept-stream. 2018-11-19 19:30:40 +02:00
cheatfate deedc37fb3 Fix test. 2018-11-19 06:42:50 +02:00
cheatfate dc482dfb45 Add simple daemon spawn test. 2018-11-19 06:34:05 +02:00
cheatfate 9262aa5e9e Add Protobuf's variable integer test. 2018-11-19 06:18:11 +02:00