vacp2p
/
nim-libp2p
mirror of https://github.com/vacp2p/nim-libp2p.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
902 Commits 154 Branches 9 Tags 110 MiB
Commit Graph

692 Commits

Author SHA1 Message Date
Dmitriy Ryajov daef00fc7b don't crash schlesi-dev 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 640c3bdc45 better exception handling and resource cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 7ff76d76b6 better exceptions 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 95774b2b81 better cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 9c2f31262e wip: try handling child stream exceptions 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov e3f8f53620 initStream method and better exceptions handling 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov d3b79b002e better exceptions and don't fail writes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 74f8b5b5f1 resource cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 93e5805c01 better exception handling 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov d83ce4c932 exceptions and resource cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov d4bdb42046 gossipsub fixes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 9d3cc9647b fix merge 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 3d9c0bffba read from stream 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 048b1db1ad revert back allread 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 8e9716f5c3 remove on transport close cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov cf76edb0dd make noise work again 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 5158d96eaf close connection on chronos close 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 46daed9a38 wip 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 0f691cbafd add eof and closed handling 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 62da2a05c3 wip: rework with proper half-closed 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 98117a3068 call write until all is written out 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 802299e69a breakout from publish loop 2020-06-02 09:10:27 -06:00
Jacek Sieka 7e31210455
cover missing case in MultiAddress.init (#198) 
* cover missing case in MultiAddress.init

* raise assert on marker in protocol

* unify inits for markers / non-markers

* fix string
 2020-06-01 14:41:45 +02:00
Jacek Sieka 88dbeebf17
add side effect annotations (#197) 2020-06-01 09:25:16 +02:00
Giovanni Petrantoni 37b98ad45c
Secure managers are now sorted, giving priority to noise (#191) 
* Secure managers are now sorted, giving priority to noise

* fix nimble test command

* Fix native tests

* fix directchat sample

* Could not write to connection - reduce verbosity

* fix interop testing

* Remove more tables

* test interop fixes

* directchat fix

* fix interop/remove some deprecation
 2020-06-01 08:41:32 +02:00
Giovanni Petrantoni 6affcda937
Less exceptions more results (#188) 
* Less exceptions more results

* Fix daemonapi and interop tests

* Add multibase

* wip multiaddress

* fix the build, consuming new result types

* fix standard setup

* Simplify match, rename into MaError, add more exaustive err text

* Fix the CI issues

* Fix directchat build

* daemon api fixes

* better err messages formatting

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
 2020-05-31 16:22:49 +02:00
Giovanni Petrantoni 7c9e5c2f7a "Could not write to connection" message split between trace and debug due to log size 2020-05-30 23:47:56 +09:00
Oskar Thorén b88bfc05f8
Make GossipSub initPubSub method public (#193) 
This means we can use it from other protocols that inherit GossipSub. Otherwise,
a lot of internal state (heartbeat lock etc) doesn't get initialized properly.
 2020-05-29 09:35:03 -06:00
Dmitriy Ryajov 7b6e1c0688
Gossipsub interop (#189) 
* interop fixes

* add custom messageid provider and fix seqno

* use ECDSA for speed

* adding messageid tests

* breakout from publish loop

* addressing review comments

* remove unneded var

* dont stop broadcasting on failed peers
 2020-05-27 12:33:49 -06:00
Giovanni Petrantoni 536555138c fix "future" typo 2020-05-26 16:08:09 +09:00
Giovanni Petrantoni 4447d97234 add back forgotten message in tryAndWarn 2020-05-26 15:21:59 +09:00
Dmitriy Ryajov 9132f16927
gossipsub fixes (#186) 2020-05-21 14:24:20 -06:00
Dmitriy Ryajov ba53c08b3c
Track incoming connections (#181) 
* call write until all is written out

* wip: rework with proper half-closed

* add eof and closed handling

* wip

* close connection on chronos close

* don't use read

* make noise work again

* don't reraise just yet

* fixes after backporting

* remove on transport close cleanup

* revert back allread

* rust interop fixes

* read from stream

* inc count before closing

* rebasing master

* store incomming connections

* fix merge

* remove unneeded changes

* use internal close flag to indicate disposal
 2020-05-21 11:33:48 -06:00
Dmitriy Ryajov 7900fd9f61
Half closed (#174) 
* call write until all is written out

* add comments to lpchannel fields

* add an eof flag to signal which end closed

* wip: rework with proper half-closed

* add eof and closed handling

* propagate closes to piped

* call parent close

* moving bufferstream trackers out

* move writeLock to bufferstream

* move writeLock out

* remove unused call

* wip

* rebasing master

* fix mplex tests

* wip

* fix bufferstream after backport

* wip

* rename to differentiate from chronos tracker

* close connection on chronos close

* make reset request asyncCheck

* fix channel cleanup

* misc

* don't use read

* fix backports

* make noise work again

* proper exception handling

* don't reraise just yet

* add convenience templates

* dont double wrap

* use async pragma

* fixes after backporting

* muxer owns connection

* remove on transport close cleanup

* revert back allread

* adding some todos

* read from stream

* inc count before closing

* rebasing master

* rebase master

* use correct exception type

* use try/finally insted of defer

* fix compile in trace mode

* reset channels on mplex close
 2020-05-19 18:14:15 -06:00
Dmitriy Ryajov 681991ae48
reduce buffer size in lpchannel (#184) 2020-05-19 16:15:36 -06:00
Giovanni Petrantoni 01339c991f
Don't use and expose directly secp types (#183) 
* Don't use and expose directly secp types

* Reuse same secp type names
 2020-05-19 14:48:55 +02:00
Giovanni Petrantoni c219100e64
Use results and no exceptions in chacha and curve25519 (#182) 2020-05-19 10:22:49 +02:00
Dmitriy Ryajov f8029e7359
use sha256 digest as cache keys (#135) 
* use sha256 digest as cache keys

* rebasing master
 2020-05-18 14:49:49 -06:00
Dmitriy Ryajov 9cf1fd0216
remove generic constructor and expose serverflags (#176) 
* remove generic constructor and expose serverflags

* fix transport constructor

* fix merge issues
 2020-05-18 13:04:05 -06:00
Dmitriy Ryajov 773b738c12
don't track Connection, track StreamTransport (#177) 
* don't track Connection, track StreamTransport

* make tests more deterministic
 2020-05-18 11:05:34 -06:00
Dmitriy Ryajov 5583168965
Track all connections (#175) 
* adding to do for future refactor

* no mix conn and transport stopping or it will race

* don't use intermediary vars
 2020-05-18 09:08:10 -06:00
Dmitriy Ryajov 1819502fb5
Cleanup - tests and logging (#178) 
* make async for proper exception handling

* tryAndWarn msg messes up Exception msg

* misc: comment out tracker dumps

* cleanup mplex tests

* more informative errors

* give CI time to run

* revert change, bacause it causes races
 2020-05-18 07:49:49 -06:00
Giovanni Petrantoni 7dcb807f64
Crypto utilities resultification (#150) 2020-05-18 07:25:55 +02:00
Dmitriy Ryajov 167f42ed45
Remove read (#171) 
* use readExactly

* remove `read`

* remove read

* no more `read`
 2020-05-14 22:02:05 -06:00
Jacek Sieka 69abf5097d
handle a few exceptions (#170) 
* handle a few exceptions

Some of these are maybe too aggressive, but in return, they'll log
their exception - more refactoring needed to sort this out - right now
we get crashes on unhandled exceptions of unknown origin

* during connection setup
* while closing channels
* while processing pubsubs

* catch exceptions that are raised and don't try to catch exceptions that are not raised

* propagate cancellederror

* one more

* more

* more

* make interop tests less fragile

* Raise expiration time in gossipsub fanout test for slow CI

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
 2020-05-14 21:56:56 -06:00
Giovanni Petrantoni 005e088405
Properly track and close mplex handlers (#166) 
* Properly track and close mplex handlers

* Avoid verbose warnings

* Fix tryAndWarn trace issue

* Handle LPEOF in lpchannel close
 2020-05-12 14:45:32 +02:00
Jacek Sieka 618e01eba3
don't crash on double peerinfo close (#167) 2020-05-11 21:05:24 +02:00
Jacek Sieka 3053f03814 fix varint issues 
* fixes #111
 2020-05-11 09:12:23 -06:00
Jacek Sieka 87e1f3c61f
fix trace logging 2020-05-09 13:56:07 +02:00
Jacek Sieka fbf640794b
move channel size constant 2020-05-09 08:48:26 +02:00
Dmitriy Ryajov 6196d56fc2
check for nil observers 2020-05-08 15:44:18 -06:00
Jacek Sieka ccd019b328
use stream directly in chronosstream (#163) 
* use stream directly in chronosstream

for now, chronos.AsyncStream is not used to provide any features on top
of chronos.Stream, so in order to simplify the code, chronosstream can
be used directly.

In particular, the exception handling is broken in the current
chronosstream - opening and closing the stream is simplified this way as
well.

A future implementation that actually takes advantage of the AsyncStream
features would wrap AsyncStream instead as a separate lpstream
implementation, leaving this one as-is.

* work around chronos exception type issue
 2020-05-08 22:10:06 +02:00
Giovanni Petrantoni c889224012 Add PubSub observer+ hooks (they can modify as well) 2020-05-08 13:31:52 -06:00
Ștefan Talpalaru 268253ea18 remove Chronos type from public API 2020-05-08 13:23:36 -06:00
Ștefan Talpalaru 313f9b0952 use a Transport.serverFlags attribute 2020-05-08 13:23:36 -06:00
Ștefan Talpalaru c480e65055 use set[ServerFlags] params instead of hardcoded flags 2020-05-08 13:23:36 -06:00
Ștefan Talpalaru 125843af7d TcpTransport.listen(): enable SO_REUSEADDR 
in order to avoid failures when restarting the process while the OS is
keeping open sockets from previous runs
 2020-05-08 13:23:36 -06:00
Jacek Sieka 1efada474c
remove readLoop in secure protocols (#162) 
* remove readLoop in secure protocols, fix security issues

* fix Defect on remote sending 0-byte noise/secio message
* remove msglen from `write` (unused)
* simplify SecureConn data flow
* document some control-flow issues

* unify exception behaviour across noise and secio

* secio would not raise on mac/decryption errors

* fix compile error
 2020-05-07 14:37:46 -06:00
Jacek Sieka 330da51819
removals (#159) 
* remove unused stream methods
* reimplement some of them with proc's
* remove broken tests
* Error->Defect for defect
* warning fixes
 2020-05-06 18:31:47 +02:00
Dmitriy Ryajov 6da4d2af48
Pubsub signatures flags (#161) 
* add verify signature flag

* add sign flag to enable/disable msg signing

* moving internal tests out to their own file

* cleanup nimble file

* remove unneeded tests

* move pubsub tests out

* fix tests
 2020-05-06 11:26:08 +02:00
Jacek Sieka e1928456a7 avoid newlines in $ 
they mess with debug prints and logging (same reason why $(seq) doesn't
print them
 2020-04-28 10:59:53 -06:00
cheatfate 290ba712e9
Fix MultiAddress.protoAddress() bug for fixed arrays. 
Add tests for it.
 2020-04-28 14:43:44 +03:00
Giovanni Petrantoni 8a22c073c7 Fix secure/noise securing explicitly, added noise to pubsub tests 2020-04-24 14:33:08 -06:00
cheatfate 917b5f5c84 Add MultiAddress.init(integer) for tcp,udp,dccp,sctp protocols. 
Add tests for it.
 2020-04-23 08:10:17 -06:00
Giovanni Petrantoni 1c4d72f5e3
Use Result construct in minasn1 (#144) 2020-04-23 14:10:20 +02:00
Giovanni Petrantoni 6ae92eb21a Hotfix noise read usage, replace with readExactly 2020-04-22 23:36:59 +09:00
Giovanni Petrantoni d96372f820 Minor tweaking in errors.nim in order to give priority to Defect 2020-04-21 22:10:47 +09:00
Giovanni Petrantoni 4c6a123d31
Add chronos trackers and used them to sanitize resource disposal (#131) 
* Add chronos trackers and used them to sanitize resource disposal

* Chronos trackers for transport tests wip

* No more chronos leaks in testtransport

* Make tcp transport and test more robust when closing

* Test async leaking tracking wip

* Fix a regression in wire connect

* Add chronos trackers to more tests and sanitize resource closure

* Wip fixing floodsub tests

* Floodsub wip

* Made floodsub basically deterministic, hit a nim bug with captures tho

* Wrap up floodsub tests refactor

* Wrapping up

* Add allFuturesThrowing utility

* Fix missing allFuturesThrowing in noise tests!

* Make tests green

* attempt fixing gossipsub failing cases

* Make sure to check also fanout in waitSub

* More verbose traces

* Gossipsub test improvments

* Refactor TcpTransport remove asyncCheck

* Add Connection trackers

* Add stricter connection tracking, wip mplex fix

* More asynccheck removal, in order to avoid connection leaks

* bump chronicles requirement

* Enable tracker dump to check CI output

* Wait for more futures in testmplex

* Remove tracker dump messages

* add tryAndWarn utility, fix mplex issue with go interop

* All allFuturesThrowing to directchat too

* make sure to cleanup on transport close
 2020-04-21 10:24:42 +09:00
Jacek Sieka e8b33c64fa
secp: use upstream secp convenience api (#141) 
* secp: use upstream secp convenience api
 2020-04-17 12:51:13 +02:00
Ștefan Talpalaru eaa73ae6e8
add stream metrics (#136) 
* add stream metrics

- just BufferStream and Connection are tracked, for now
- flag checking is enforced more strictly in close(), since it became
  clear that instances are closed multiple times

* add "metrics" dependency

and sort the list
 2020-04-14 15:27:07 +02:00
Ștefan Talpalaru 7723403b1f
debug prints (#132) 
* debug prints

* CI: enable stack traces

* Azure: better NimBinaries cache key

* CI changes

- Azure: remove Linux target
- Travis: add ARM64 target

* uglify the code in order to save 12 bytes per LPStream object
 2020-04-14 15:21:16 +02:00
Jacek Sieka 2b823bde68
secp: update (#138) 2020-04-12 19:03:08 +02:00
Giovanni Petrantoni 303ec297da
Start removing allFutures (#125) 
* Start removing allFutures

* More allfutures removal

* Complete allFutures removal except legacy and tests

* Introduce table values copies to prevent error

* Switch to allFinished

* Resolve TODOs in flood/gossip

* muxer handler, log and re-raise

* Add a common and flexible way to check multiple futures
 2020-04-11 13:08:25 +09:00
Dmitriy Ryajov f4740c8b8e fix trace runs in connection 2020-04-07 14:55:05 -06:00
Dmitriy Ryajov 00fbc9246e fix nil condition 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 6cbcc7859e reduse usssage of asyncCheck 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov bd49a35e0a formatting 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 976164ba3c proper connection cleanup 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 0c8dde15e7 remove unused field 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 4ee1b4a66d connection closing tests 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 5a657c9264 disable storing connections on internal table 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 87b35de83e close connections 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov bae4713a85 small cleanup 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 8dda6c289d more connection closes to fix leaks 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov e4303110a6 properly close connections 2020-04-07 12:16:59 -06:00
Giovanni Petrantoni 35a48fa560 Re-enable gossipsub internal tests when running CI 
minor bonus: add a link in the comments about bearssl issue with
callbacks
 2020-04-07 22:07:00 +09:00
Giovanni Petrantoni a576069305 Chachapoly new bearssl fixes 2020-04-07 17:52:19 +09:00
Giovanni Petrantoni f7af2c2e5c Fix possible crash in tcptransport when server is nil 2020-04-06 12:56:17 +09:00
Giovanni Petrantoni 3514733060 Fix table assertion, edited while iterating 
(the fix is not so nice.. adds plenty of allocations, but for now should
be ok)
 2020-04-05 01:19:10 +09:00
Giovanni Petrantoni e39bf0a4cf
Mplex: Add the ability to send any size payload (#123) 
* Mplex: Add the ability to send any size payload

* Ensure size of coder header
 2020-04-03 17:26:46 +02:00
Ștefan Talpalaru 7f8090b166
SecioConn.readMessage(): fix the exception types 2020-04-02 19:07:14 +02:00
Dmitriy Ryajov 5285f0d091
Fix/misc (#116) 
* only check for payload size

* only subscribe if connection succeeded

* fix failing test

* check that the strem is active before openning

* msg type should not be > than 0x7

* fix tests

* check max against enum val
 2020-03-29 08:28:48 -06:00
Dmitriy Ryajov a35ecc6227 reverting connection changes 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov 3effb95f10 close underlying bufferstream in lpchannel 2020-03-28 09:29:43 -06:00
Giovanni Petrantoni 56392913f9
Allow to specify a read size in readLp (#117) 
* Allow to specify a read size in readLp

Also fix secio fragments + test

* Revert readLp changes, use just `read` in test
 2020-03-26 15:06:47 +09:00
Giovanni Petrantoni 33ceefb209 Remove nim random usage from noise 2020-03-26 15:04:46 +09:00
Giovanni Petrantoni b132eae939 Add secio oversized payload fragmentation 
Cannot be used because Connection Read size hard limit tho
 2020-03-25 16:53:35 +09:00
Giovanni Petrantoni 1550bea1cc Support arbitrary stream write sizes in noise 2020-03-25 16:10:11 +09:00
Dmitriy Ryajov 4199508f17 readMplexVarint should return uint64 2020-03-24 15:41:40 -06:00
Dmitriy Ryajov 2de98751ae fix: use exact types for mplex id 2020-03-24 15:41:40 -06:00
cheatfate 1f5d994700
Fix compilation errors introduced by latest chronos. 2020-03-24 09:48:05 +02:00
Giovanni Petrantoni 3d10513b3e Properly await the handshake end in secure handler 2020-03-24 15:34:02 +09:00
Giovanni Petrantoni 0a3e4a764b
Less verbose traces (#112) 
* Make traces less verbose with shortHexDump utility

* Rename shortHexDump into shortLog

* Improve shortLog, add shortLog for crypto keys

* Add proper shortLog implementations in messages
 2020-03-23 15:03:36 +09:00
Giovanni Petrantoni b1a34f478e Avoid 2 async consecutive writes in noise HS 2020-03-18 20:40:51 +09:00
Giovanni Petrantoni c02fca25f8
Noise (#90) 
* Start ChaCha20Poly1305 integration (BearSSL)

* Add Curve25519 (BearSSL) required operations for noise

* Fix curve mulgen iterate/derive

* Fix misleading header

* Add chachapoly proper test

* Curve25519 integration tests (failing, something is wrong)

* Add few converters, finish c25519 integration tests

* Remove implicit converters

* removed internal globals

* Start noise implementation

* Fix public() using proper bear mulgen

* Noise protocol WIP

* Noise progress

* Add a quick nim version of HKDF

* Converted hkdf to iterator, useful for noise

* Noise protocol implementation progress

* Noise progress

* XX handshake almost there

* noise progress

* Noise, testing handshake with test vectors

* Noise handshake progress, still wrong somewhere!

* Noise handshake success!

* Full verified noise XX handshake completed

* Fix and rewrite test to be similar to switch one

* Start with connection upgrade

* Switch chachapoly to CT implementations

* Improve HKDF implementation

* Use a type insted of tuple for HandshakeResult

* Remove unnecessary Let

* More cosmetic fixes

* Properly check randomBytes result

* Fix chachapoly signature

* Noise full circle (altho dispatcher is nil cursed)

* Allow nil aads in chachapoly routines

* Noise implementation up to running full test

* Use bearssl HKDF as well

* Directly use bearssl rng for curve25519 keys

* Add a (disabled/no CI) noise interop test server

* WIP on fixing interop issues

* More fixes in noise implementation for interop

* bump chronos requirement (nimble)

* Add a chachapoly test for very small size payloads

* Noise, more tracing

* Add 2 properly working noise tests

* Fix payload packing, following the spec properly (and not go version but
rather rust)

* Sanity, replace discard with asyncCheck

* Small fixes and optimization

* Use stew endian2 rather then system endian module

* Update nimble deps (chronos)

* Minor cosmetic/code sanity fixes

* Noise, handle Nonce max

* Noise tests, make sure to close secured conns

* More polish, improve code readability too

* More polish and testing again which test fails

* Further polishing

* Restore noise tests

* Remove useless Future[void]

* Remove useless CipherState initializer

* add a proper read wait future in second noise test

* Remove noise generic secure implementation for now

* Few fixes to run eth2 sim

* Add more debug info in noise traces

* Merge size + payload write in sendEncryptedMessage

* Revert secure interface, add outgoing property directly in newNoise

* remove sendEncrypted and receiveEncrypted

* Use openarray in chachapoly and curve25519 helpers
 2020-03-17 13:30:01 +01:00
Dmitriy Ryajov 761b221b75 don't trace the payload 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 5c234f704d added 1 byte jitter test 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 406b79887d add jitter tolerance to mplex 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 82af623641 use correct shift 2020-03-11 09:58:24 -06:00
Dmitriy Ryajov 07695b2260 use `readExactly` not `read` to read all bytes 2020-03-11 09:58:24 -06:00
Jacek Sieka 52cb7ced04
prefer endians2 (#98) 2020-03-10 21:40:52 +01:00
Dmitriy Ryajov df29ac760e remove missing include 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov fbe54e5c5a move SecureConn to secure.nim 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov 19095a0f85 move `secure` to base as well 2020-03-09 13:26:38 -06:00
Eugene Kabanov d1c6591b8a Fix and refactoring of some procedures which are able to return nil as result (#97) 
* Fix do not return nil as result.

* Fix mplex test to properly raise.
 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov c3e791db0d remove unused imports 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov f1a297f0f3 add initiator flag to handshake 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov d1d133319e change handleConn to be a proc 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov 9aa9e97602 feat: move connection handling logic to base class 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov 871d8d7478 remove white space 2020-03-09 13:26:38 -06:00
Eugene Kabanov 5701d937c8
Signed variable integers fixes. (#96) 
* Fix signed varints.
Add tests for signed varints.
Remove some casts to allow usage at compile time.

* Fix vsizeof() on 32bit platforms.

* Add `hint` and `zint` types for proper signed integer encoding.

* Fix varint related bugs.

* Update requirements.

* Fix interop tests because of fixed readLine.

* Add putVarint, getVarint and tests.
 2020-03-06 20:19:43 +01:00
Eugene Kabanov 381630f185
Fix and refactoring of some procedures which are able to return nil as result (#97) 
* Fix do not return nil as result.

* Fix mplex test to properly raise.
 2020-03-04 21:45:14 +02:00
Dmitriy Ryajov ffc3b04222 read should await for data on an empty buff 2020-03-01 04:06:42 -06:00
Eugene Kabanov 91d2f52f4e
Add some utility procedures to MultiAddress. (#94) 
* Add some utility procedures to MultiAddress.

* Do not use result and add more operations to be more effective for compiler.
 2020-02-27 19:26:42 +02:00
Dmitriy Ryajov 85d4baca95 await connection 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 946b9580c0 don't use nil result 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 52c270dcf9 warn on failed pubsub subscribe 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov eb49d4b218 no empty proto dials and add connect method 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 9efc08cb2f fix: #83 2020-02-24 08:58:02 -06:00
Dmitriy Ryajov 6a7f9f058c address review comments 2020-02-21 09:21:06 -06:00
Dmitriy Ryajov fbcef69891 implicitelly dial pubsub if enabled 2020-02-21 09:21:06 -06:00
Giovanni Petrantoni cb156f3260 Fix public() using proper bear mulgen 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 93a480e6f3 removed internal globals 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni e0e371994c Remove implicit converters 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni f039853a19 Add few converters, finish c25519 integration tests 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 99a169b65d Curve25519 integration tests (failing, something is wrong) 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 2103aa59f3 Add chachapoly proper test 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 6ba274de27 Fix misleading header 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni f47fb628f0 Fix curve mulgen iterate/derive 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 8dec368878 Add Curve25519 (BearSSL) required operations for noise 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 71998ffc67 Start ChaCha20Poly1305 integration (BearSSL) 2020-02-18 09:35:14 -06:00
Dmitriy Ryajov 1a987a9c5b fix: delete mplex channel 2020-02-16 13:51:03 -06:00
Dmitriy Ryajov 65ed2c7cee make code more robust 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 9023bf786d remove sleeps 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov acdaeb8f5d working out synchronization issues 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov b1dd564c32 make sure we don't exit the handler prematurely 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f9ad113d11 don't send close message if remote closed 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f767614827 remove unnecesary async lock 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 934c858542 increase timeout to allow floodsub to finish 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 26a4510377 remove unnecesary nil check in peerinfo 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 0349165830 typo 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 7f8eb0272e cleanup and fix tests 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 6316b07e81 don't crash when stringifying nil peerinfo 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 4faf35c52c throw exception if handshake failed 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f4e9bc8bfb remove sleepAsync for synchronization 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 53e163abf2 check for size bounds when reading varints 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 94fc4e6fd2 don't use sleeps for synchronization 2020-02-16 11:31:35 -06:00
Giovanni Petrantoni 39dc9ad8a3 Revert "Implement timeout in mplex in order to read reliably close state" 
This reverts commit 314cb6e6bf.
 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 11bd55308f Revert "Properly use already specified MaxReadWriteTime as timeout" 
This reverts commit 8f5dd75e7f.
 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni ea0d167285 Fix max size write mplex test 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni cf5dd27e57 Properly use already specified MaxReadWriteTime as timeout 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 4034dfae19 Implement timeout in mplex in order to read reliably close state 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 136ac3f550 Use already existing error for channel push over size 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni f9cbdc252f Improve connection.nim read error clarity when buffer overflow 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 120ba0d528 Add a safety check on message size limit when pushing new data in mplex 2020-02-12 11:57:38 -05:00
Giovanni Petrantoni 3531ebf772 Fix typo 2020-02-12 11:57:38 -05:00
cheatfate 5b6f4bf5fa
Export MultiAddress pattern procedures. 2020-02-12 12:04:12 +02:00
Eugene Kabanov 540e79a430
Add peer lifetime feature for PeerInfo. (#77) 
* Add peer lifetime feature for PeerInfo.
Refactor peerinfo to use openarrays instead of sequences.
Fix tests and examples to use arrays instead of sequences.

* Add access to lifetime Future[T] itself.
 2020-02-11 12:53:39 -05:00
Giovanni Petrantoni 23712ecf3b
Lazy channels (#78) 
* Implemented lazy stream opening for mplex connections

* Properly fix newStream usage

* Make lazy channel open optional

* Add Lazy channel test

* Cleanup mplex test

* Move lazyness properly into LPChannel

* Connection writeLp back to proc
 2020-02-11 12:30:36 -05:00
Dmitriy Ryajov 88a030d8fb fix: removing timeouts from conn 2020-02-05 20:38:43 +01:00
Dmitriy Ryajov 2232ca598e don't timeout in pubsub 2020-02-04 17:59:57 +01:00
Zahary Karadjov 1bd933cd5a
More precise tracing 2020-02-04 17:27:32 +01:00
Zahary Karadjov 7bd305471c
Make sure the library can compile with json logging in trace mode 2020-02-04 15:17:39 +01:00
Dmitriy Ryajov 691efaa8a1 dont use result 2020-01-30 09:30:08 -06:00
Dmitriy Ryajov 171d60fb52 fix: reraise exception in readLp 2020-01-30 09:30:08 -06:00
cheatfate 06886290dd
Add links to specifications to varint comments. 2020-01-29 11:14:25 +02:00
Eugene Kabanov d902127595 Fix LibP2P varint implementation to follow latest specification. (#65) 
* Fix LibP2P varint implementation to decode only minimal valus.
 2020-01-28 16:29:42 +01:00
Dmitriy Ryajov 007b0219f3 use ECDSA as defaults, faster in tests 2020-01-09 12:55:21 -06:00
Dmitriy Ryajov 667691f784 send messages in batches 2020-01-09 12:55:21 -06:00
Dmitriy Ryajov cd8961cfb9 close and cleanup connections 2020-01-09 12:55:21 -06:00
Dmitriy Ryajov c6561b8851 add timeouts to connection/secio 2020-01-09 12:55:21 -06:00
cheatfate 107e71203d Fix #61. 2020-01-09 17:38:10 +02:00
Dmitriy Ryajov 58cdefac21 don't expose private api 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 0fb1f1c5b8 strenghten pubsub interop testing 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 8714c66353 allow empty protocol string in dial 2019-12-24 10:35:35 -06:00
Dmitriy Ryajov 68cc57669e
Feat/pubsub validators (#58) 
* feat: adding validator hooks to pubsub

* expose add/remove validators on switch

* do less unnecessary copyng
 2019-12-16 23:24:03 -06:00
Dmitriy Ryajov b6b0cdea98
fix: inconsistent error handling (#56) 
* fix: inconsistent error handling

* fixing a few more returns

* remov reduntant returns

* feat: dont sleep in readLoop
 2019-12-13 09:25:07 -06:00
Yuriy Glukhov e0aae6d8ac Moved base32, 58 and 64 to stew 2019-12-13 16:00:39 +02:00
Dmitriy Ryajov 293a219dbe
Cleanup (#55) 
* fix: don't allow replacing pubkey

* fix: several small improvements

* removing pubkey setter

* improove error handling

* remove the use of Option[T] if not needed

* don't use optional

* fix-ci: temporarily pin p2pd to a working tag

* fix example to comply with latest changes

* bumping p2pd again to a higher version
 2019-12-10 14:50:35 -06:00
Zahary Karadjov 77e866d29a
Address review comments 2019-12-09 19:17:29 +02:00
Zahary Karadjov 454f658ba8
Fixes and tweaks related to the beacon node integration 
* Bugfix: Dialing an already connected peer may lead to crash

* Introduced a standard_setup module allowing to instantiate
  the `Switch` object in an easier manner.

* Added `Switch.disconnect(peer)`

* Trailing space removed (sorry about polluting the diff)
 2019-12-08 23:58:43 +02:00
Dmitriy Ryajov 5f6fcc3d90
extract public and private keys fields from peerid (#44) 
* extract public and private keys fields from peerid

* allow assigning a public key

* cleaned up TODOs

* make pubsub prefix a const

* public key should be an `Option`
 2019-12-07 10:36:39 -06:00
Dmitriy Ryajov e623e70e7b
PubSub (Gossip & Flood) Implementation (#36) 
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon. 

* add close event

* wip: gossipsub

* splitting rpc message

* making message handling more consistent

* initial gossipsub implementation

* feat: nim 1.0 cleanup

* wip: gossipsub protobuf

* adding encoding/decoding of gossipsub messages

* add disconnect handler

* add proper gossipsub msg handling

* misc: cleanup for nim 1.0

* splitting floodsub and gossipsub tests

* feat: add mesh rebalansing

* test pubsub

* add mesh rebalansing tests

* testing mesh maintenance

* finishing mcache implementatin

* wip: commenting out broken tests

* wip: don't run heartbeat for now

* switchout debug for trace logging

* testing gossip peer selection algorithm

* test stream piping

* more work around message amplification

* get the peerid from message

* use timed cache as backing store

* allow setting timeout in constructor

* several changes to improve performance

* more through testing of msg amplification

* prevent gc issues

* allow piping to self and prevent deadlocks

* improove floodsub

* allow running hook on cache eviction

* prevent race conditions

* prevent race conditions and improove tests

* use hashes as cache keys

* removing useless file

* don't create a new seq

* re-enable pubsub tests

* fix imports

* reduce number of runs to speed up tests

* break out control message processing

* normalize sleeps between steps

* implement proper transport filtering

* initial interop testing

* clean up floodsub publish logic

* allow dialing without a protocol

* adding multiple reads/writes

* use protobuf varint in mplex

* don't loose conn's peerInfo

* initial interop pubsub tests

* don't duplicate connections/peers

* bring back interop tests

* wip: interop

* re-enable interop and daemon tests

* add multiple read write tests from handlers

* don't cleanup channel prematurely

* use correct channel to send/receive msgs

* adjust tests with latest changes

* include interop tests

* remove temp logging output

* fix ci

* use correct public key serialization

* additional tests for pubsub interop
 2019-12-05 20:16:18 -06:00
Dmitriy Ryajov 903e79ede1
Feat/conn cleanup (#41) 
Backporting proper connection cleanup from #36 to align with latest chronos changes.

* add close event

* use proper varint encoding

* add proper channel cleanup in mplex

* add connection cleanup in secio

* tidy up

* add dollar operator

* fix tests

* don't close connections prematurely

* handle closing streams properly

* misc

* implement address filtering logic

* adding pipe tests

* don't use gcsafe if not needed

* misc

* proper connection cleanup and stream muxing

* re-enable pubsub tests
 2019-12-03 22:44:54 -06:00
Dmitriy Ryajov 1df16bdbce set log level to trace - not enabled by default 2019-12-02 18:43:21 -06:00
Zahary Karadjov f9eed172d4 Don't handle Exception 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov 26366f8b56 use `Exception` to emulate `getCurrentException()` 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov a5b051a14f fix: use exception `msg` field in logs 2019-12-03 01:26:59 +02:00
Dmitriy Ryajov 9e437a681b fix: revert to proper exception handling 2019-12-03 01:26:59 +02:00
cheatfate dde8c01448
Workaround for https://github.com/nim-lang/Nim/issues/12764 . 
Remove usage of getCurrentException().
 2019-11-28 20:58:56 +02:00
Dmitriy Ryajov fe3c1d45bb misc: removing unneded accessors 2019-11-06 12:13:21 -06:00
Ștefan Talpalaru f3fc763895
daemonapi: direct p2pd's output to parent's streams 2019-11-05 21:16:56 +01:00
cheatfate 76fee97724
Fix MultiAddress.init(PeerID) compilation problem. 2019-11-04 20:57:17 +02:00
cheatfate 833e8c9406
Fix overflow problem. 
Fix compilation warnings.
 2019-10-29 20:51:48 +02:00
Dmitriy Ryajov 400218ba01 fixing remaining overflows in BE/LE 2019-10-29 11:02:28 -06:00
Dmitriy Ryajov 541f0f2a41 avoid buffer overflow on BE/LE conversions 2019-10-29 10:10:19 -06:00
cheatfate cc0d45c482
Add protoAddress() utility procedure. 2019-10-29 15:38:37 +02:00
Dmitriy Ryajov 21af081066 remove init in favor of upstream init 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a4431a86a1 fixing windows build issue 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 0b77c8c6e3 add init from TransportAddress to multiaddress 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov ae7b354819 unused directory 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9e16fd020e use PeerID `==` for comparisons 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 018da65708 compare peer string ids, rather than keys 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3194e3ac5b fixes after building with 0.20.2 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bb277d4b56 readLp should warn on ignored exceptions 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 464d75d159 minor style fixes 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2066e81658 set default timeout to 10 secs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 1d4b51413e option to allow triggering own handlers on publish 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 37d7a03fba use a timed cache in floodsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov e98ec9fd49 use doAssert for release asserts 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f190c155d3 don't throw on missing peer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3b9d34116d decrease floodsub traffic 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f9a04dbb9d todo: poll should allow for timeouts 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cfd31c5251 properly handle empty messages 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6cfbf2c124 don't send messages to self 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov a01d1db6a0 prperly close and chronos connections 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 968fd85d49 properly exit readLoop on error/stream end 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 34d1a641de cleanup/test pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f8872dd51c make log topics consistent 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 3c5b77affa correctly identify incoming connection 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 68eb142a7c end multiplex read loop on exception 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov cc2072d290 always cleanup connection on exit 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov c7606ce2eb readMsg should never return an empty buffer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f7e5c8ee30 bring back commented tests 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov bc46a76029 connection should raise on invalid varint 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 15bae7bec5 use `result` instead of buffer 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f5508be544 yield to chronos in mplex loop 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b5c4eaf9b8 don't use result in oneliners 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 06b759e049 review cleanup 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5ae37cac98 no more buffer resets on len prefixed reads 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov dd3f85965b should not handle transport exceptions 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 99869aa8b8 don't reset buffer on exception 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 6baf089bb4 move tomultiaddr to wire.nim 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov f3afe0a6ce feat: allow ussage of 0 addrs 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 9862064234 changed copyright year 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 5b3f93ba1c feat: allow multiple handlers per topic in pubsub 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 2f31fc6940 misc: TODO for timeouts and message limit in mplex 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov b270515bb3 feat: make private/public keys Option[T] 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 663ce6c589 misc: nimpretty 2019-10-11 08:15:24 +09:00
Dmitriy Ryajov 054085620c logging: switch debug for trace in most cases 2019-10-11 08:15:24 +09:00