Commit Graph

240 Commits

Author SHA1 Message Date
Dmitriy Ryajov 20c68a2018 use all() for futures and track connections 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 6112de746d remove unneeded changes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 5f704e6825 rust interop fixes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 8e48bb117d rebasing master 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 77017d1e7e inc count before closing 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 8963c2d0ed fixes after backporting 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov bfe84816ac don't reraise just yet 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 87e1cf924f don't use read 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 46daed9a38 wip 2020-06-02 09:10:27 -06:00
Giovanni Petrantoni 37b98ad45c
Secure managers are now sorted, giving priority to noise (#191)
* Secure managers are now sorted, giving priority to noise

* fix nimble test command

* Fix native tests

* fix directchat sample

* Could not write to connection - reduce verbosity

* fix interop testing

* Remove more tables

* test interop fixes

* directchat fix

* fix interop/remove some deprecation
2020-06-01 08:41:32 +02:00
Giovanni Petrantoni 6affcda937
Less exceptions more results (#188)
* Less exceptions more results

* Fix daemonapi and interop tests

* Add multibase

* wip multiaddress

* fix the build, consuming new result types

* fix standard setup

* Simplify match, rename into MaError, add more exaustive err text

* Fix the CI issues

* Fix directchat build

* daemon api fixes

* better err messages formatting

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2020-05-31 16:22:49 +02:00
Dmitriy Ryajov 7b6e1c0688
Gossipsub interop (#189)
* interop fixes

* add custom messageid provider and fix seqno

* use ECDSA for speed

* adding messageid tests

* breakout from publish loop

* addressing review comments

* remove unneded var

* dont stop broadcasting on failed peers
2020-05-27 12:33:49 -06:00
Dmitriy Ryajov 9132f16927
gossipsub fixes (#186) 2020-05-21 14:24:20 -06:00
Dmitriy Ryajov ba53c08b3c
Track incoming connections (#181)
* call write until all is written out

* wip: rework with proper half-closed

* add eof and closed handling

* wip

* close connection on chronos close

* don't use read

* make noise work again

* don't reraise just yet

* fixes after backporting

* remove on transport close cleanup

* revert back allread

* rust interop fixes

* read from stream

* inc count before closing

* rebasing master

* store incomming connections

* fix merge

* remove unneeded changes

* use internal close flag to indicate disposal
2020-05-21 11:33:48 -06:00
Dmitriy Ryajov 7900fd9f61
Half closed (#174)
* call write until all is written out

* add comments to lpchannel fields

* add an eof flag to signal which end closed

* wip: rework with proper half-closed

* add eof and closed handling

* propagate closes to piped

* call parent close

* moving bufferstream trackers out

* move writeLock to bufferstream

* move writeLock out

* remove unused call

* wip

* rebasing master

* fix mplex tests

* wip

* fix bufferstream after backport

* wip

* rename to differentiate from chronos tracker

* close connection on chronos close

* make reset request asyncCheck

* fix channel cleanup

* misc

* don't use read

* fix backports

* make noise work again

* proper exception handling

* don't reraise just yet

* add convenience templates

* dont double wrap

* use async pragma

* fixes after backporting

* muxer owns connection

* remove on transport close cleanup

* revert back allread

* adding some todos

* read from stream

* inc count before closing

* rebasing master

* rebase master

* use correct exception type

* use try/finally insted of defer

* fix compile in trace mode

* reset channels on mplex close
2020-05-19 18:14:15 -06:00
Dmitriy Ryajov 9cf1fd0216
remove generic constructor and expose serverflags (#176)
* remove generic constructor and expose serverflags

* fix transport constructor

* fix merge issues
2020-05-18 13:04:05 -06:00
Dmitriy Ryajov 773b738c12
don't track Connection, track StreamTransport (#177)
* don't track Connection, track StreamTransport

* make tests more deterministic
2020-05-18 11:05:34 -06:00
Dmitriy Ryajov 1819502fb5
Cleanup - tests and logging (#178)
* make async for proper exception handling

* tryAndWarn msg messes up Exception msg

* misc: comment out tracker dumps

* cleanup mplex tests

* more informative errors

* give CI time to run

* revert change, bacause it causes races
2020-05-18 07:49:49 -06:00
Giovanni Petrantoni 7dcb807f64
Crypto utilities resultification (#150) 2020-05-18 07:25:55 +02:00
Dmitriy Ryajov 167f42ed45
Remove read (#171)
* use readExactly

* remove `read`

* remove read

* no more `read`
2020-05-14 22:02:05 -06:00
Jacek Sieka 69abf5097d
handle a few exceptions (#170)
* handle a few exceptions

Some of these are maybe too aggressive, but in return, they'll log
their exception - more refactoring needed to sort this out - right now
we get crashes on unhandled exceptions of unknown origin

* during connection setup
* while closing channels
* while processing pubsubs

* catch exceptions that are raised and don't try to catch exceptions that are not raised

* propagate cancellederror

* one more

* more

* more

* make interop tests less fragile

* Raise expiration time in gossipsub fanout test for slow CI

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-05-14 21:56:56 -06:00
Giovanni Petrantoni 100d6ef595 Raise expiration time in gossipsub fanout test for slow CI 2020-05-15 11:01:33 +09:00
Jacek Sieka 3053f03814 fix varint issues
* fixes #111
2020-05-11 09:12:23 -06:00
Jacek Sieka ccd019b328
use stream directly in chronosstream (#163)
* use stream directly in chronosstream

for now, chronos.AsyncStream is not used to provide any features on top
of chronos.Stream, so in order to simplify the code, chronosstream can
be used directly.

In particular, the exception handling is broken in the current
chronosstream - opening and closing the stream is simplified this way as
well.

A future implementation that actually takes advantage of the AsyncStream
features would wrap AsyncStream instead as a separate lpstream
implementation, leaving this one as-is.

* work around chronos exception type issue
2020-05-08 22:10:06 +02:00
Giovanni Petrantoni c889224012 Add PubSub observer+ hooks (they can modify as well) 2020-05-08 13:31:52 -06:00
Jacek Sieka 1efada474c
remove readLoop in secure protocols (#162)
* remove readLoop in secure protocols, fix security issues

* fix Defect on remote sending 0-byte noise/secio message
* remove msglen from `write` (unused)
* simplify SecureConn data flow
* document some control-flow issues

* unify exception behaviour across noise and secio

* secio would not raise on mac/decryption errors

* fix compile error
2020-05-07 14:37:46 -06:00
Jacek Sieka 330da51819
removals (#159)
* remove unused stream methods
* reimplement some of them with proc's
* remove broken tests
* Error->Defect for defect
* warning fixes
2020-05-06 18:31:47 +02:00
Dmitriy Ryajov 6da4d2af48
Pubsub signatures flags (#161)
* add verify signature flag

* add sign flag to enable/disable msg signing

* moving internal tests out to their own file

* cleanup nimble file

* remove unneeded tests

* move pubsub tests out

* fix tests
2020-05-06 11:26:08 +02:00
cheatfate 290ba712e9
Fix MultiAddress.protoAddress() bug for fixed arrays.
Add tests for it.
2020-04-28 14:43:44 +03:00
cheatfate 917b5f5c84 Add MultiAddress.init(integer) for tcp,udp,dccp,sctp protocols.
Add tests for it.
2020-04-23 08:10:17 -06:00
Giovanni Petrantoni 1c4d72f5e3
Use Result construct in minasn1 (#144) 2020-04-23 14:10:20 +02:00
Giovanni Petrantoni 4c6a123d31
Add chronos trackers and used them to sanitize resource disposal (#131)
* Add chronos trackers and used them to sanitize resource disposal

* Chronos trackers for transport tests wip

* No more chronos leaks in testtransport

* Make tcp transport and test more robust when closing

* Test async leaking tracking wip

* Fix a regression in wire connect

* Add chronos trackers to more tests and sanitize resource closure

* Wip fixing floodsub tests

* Floodsub wip

* Made floodsub basically deterministic, hit a nim bug with captures tho

* Wrap up floodsub tests refactor

* Wrapping up

* Add allFuturesThrowing utility

* Fix missing allFuturesThrowing in noise tests!

* Make tests green

* attempt fixing gossipsub failing cases

* Make sure to check also fanout in waitSub

* More verbose traces

* Gossipsub test improvments

* Refactor TcpTransport remove asyncCheck

* Add Connection trackers

* Add stricter connection tracking, wip mplex fix

* More asynccheck removal, in order to avoid connection leaks

* bump chronicles requirement

* Enable tracker dump to check CI output

* Wait for more futures in testmplex

* Remove tracker dump messages

* add tryAndWarn utility, fix mplex issue with go interop

* All allFuturesThrowing to directchat too

* make sure to cleanup on transport close
2020-04-21 10:24:42 +09:00
Jacek Sieka e8b33c64fa
secp: use upstream secp convenience api (#141)
* secp: use upstream secp convenience api
2020-04-17 12:51:13 +02:00
Dmitriy Ryajov 4ee1b4a66d connection closing tests 2020-04-07 12:16:59 -06:00
Giovanni Petrantoni e39bf0a4cf
Mplex: Add the ability to send any size payload (#123)
* Mplex: Add the ability to send any size payload

* Ensure size of coder header
2020-04-03 17:26:46 +02:00
Ștefan Talpalaru 7f8090b166
SecioConn.readMessage(): fix the exception types 2020-04-02 19:07:14 +02:00
Dmitriy Ryajov 5285f0d091
Fix/misc (#116)
* only check for payload size

* only subscribe if connection succeeded

* fix failing test

* check that the strem is active before openning

* msg type should not be > than 0x7

* fix tests

* check max against enum val
2020-03-29 08:28:48 -06:00
Dmitriy Ryajov 6bb4e91a39 fix tests 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov a92d616a3e fix tests 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov 3effb95f10 close underlying bufferstream in lpchannel 2020-03-28 09:29:43 -06:00
Giovanni Petrantoni c0a10ecea7
Disable new fragmentation tests (#119)
Merging for now to fix tests (virtually)
Could not repro locally yet
2020-03-27 10:47:30 +09:00
Giovanni Petrantoni 56392913f9
Allow to specify a read size in readLp (#117)
* Allow to specify a read size in readLp

Also fix secio fragments + test

* Revert readLp changes, use just `read` in test
2020-03-26 15:06:47 +09:00
Giovanni Petrantoni 1550bea1cc Support arbitrary stream write sizes in noise 2020-03-25 16:10:11 +09:00
cheatfate a2acdd7933
Disable unreliable provide cid test. 2020-03-24 10:05:11 +02:00
Giovanni Petrantoni c02fca25f8
Noise (#90)
* Start ChaCha20Poly1305 integration (BearSSL)

* Add Curve25519 (BearSSL) required operations for noise

* Fix curve mulgen iterate/derive

* Fix misleading header

* Add chachapoly proper test

* Curve25519 integration tests (failing, something is wrong)

* Add few converters, finish c25519 integration tests

* Remove implicit converters

* removed internal globals

* Start noise implementation

* Fix public() using proper bear mulgen

* Noise protocol WIP

* Noise progress

* Add a quick nim version of HKDF

* Converted hkdf to iterator, useful for noise

* Noise protocol implementation progress

* Noise progress

* XX handshake almost there

* noise progress

* Noise, testing handshake with test vectors

* Noise handshake progress, still wrong somewhere!

* Noise handshake success!

* Full verified noise XX handshake completed

* Fix and rewrite test to be similar to switch one

* Start with connection upgrade

* Switch chachapoly to CT implementations

* Improve HKDF implementation

* Use a type insted of tuple for HandshakeResult

* Remove unnecessary Let

* More cosmetic fixes

* Properly check randomBytes result

* Fix chachapoly signature

* Noise full circle (altho dispatcher is nil cursed)

* Allow nil aads in chachapoly routines

* Noise implementation up to running full test

* Use bearssl HKDF as well

* Directly use bearssl rng for curve25519 keys

* Add a (disabled/no CI) noise interop test server

* WIP on fixing interop issues

* More fixes in noise implementation for interop

* bump chronos requirement (nimble)

* Add a chachapoly test for very small size payloads

* Noise, more tracing

* Add 2 properly working noise tests

* Fix payload packing, following the spec properly (and not go version but
rather rust)

* Sanity, replace discard with asyncCheck

* Small fixes and optimization

* Use stew endian2 rather then system endian module

* Update nimble deps (chronos)

* Minor cosmetic/code sanity fixes

* Noise, handle Nonce max

* Noise tests, make sure to close secured conns

* More polish, improve code readability too

* More polish and testing again which test fails

* Further polishing

* Restore noise tests

* Remove useless Future[void]

* Remove useless CipherState initializer

* add a proper read wait future in second noise test

* Remove noise generic secure implementation for now

* Few fixes to run eth2 sim

* Add more debug info in noise traces

* Merge size + payload write in sendEncryptedMessage

* Revert secure interface, add outgoing property directly in newNoise

* remove sendEncrypted and receiveEncrypted

* Use openarray in chachapoly and curve25519 helpers
2020-03-17 13:30:01 +01:00
Dmitriy Ryajov 5c234f704d added 1 byte jitter test 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov dc97595aad remove irrelevant comment 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 59abd2e4cd remove unnecesary while 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 4fc84cbe81 add jitter tolerance to mplex 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 8c96de5397 reduce waiting time from 5 seconds to 500 millis in msg size test 2020-03-10 12:43:32 -06:00