vacp2p
/
nim-libp2p
mirror of https://github.com/vacp2p/nim-libp2p.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
896 Commits 153 Branches 9 Tags 116 MiB
Commit Graph

687 Commits

Author SHA1 Message Date
Giovanni Petrantoni ee6e545878
multistream select make sure to not report NA (#235) 
* multistream select make sure to not report NA but rather empty string if all fails

Also re-enable tests

* avoid using bad constructs, make multistream.select flow crystal clear
 2020-06-22 15:38:48 -06:00
Jacek Sieka 6331b04cb4
secp: requiresInit updates (#237) 
* secp: requiresInit updates

* fix
 2020-06-22 19:03:15 +02:00
Jacek Sieka b99fd88deb
logging fixes 2020-06-21 11:14:19 +02:00
Giovanni Petrantoni 7852c6dd0f
Noise and eth2/nbc fixes (#226) 
* Remove noise padding payload (spec removed it)

* add log scope in secure

* avoid defect array out of range in switch secure when "na"

* improve identify traces

* wip noise fixes

* noise protobuf adjustments (trying)

* add more debugging messages/traces, improve their actual contents

* re-enable ID check in noise

* bump go daemon tag version

* bump go daemon tag version

* enable noise in daemonapi

* interop testing, (both secio and noise will be tested)

* azure cache bump (p2pd)

* CI changes

- Travis: use Go 1.14
- azure-pipelines.yml: big cleanup
- Azure: bump cache keys
- build 64-bit p2pd on 32-bit Windows
- install both Mingw-w64 architectures

* noise logging fixes

* alternate testing between noise and secio

* increase timeout to avoid VM errors in CI (multistream tests)

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
 2020-06-20 19:56:55 +09:00
Dmitriy Ryajov 7a1c1c2ea6
fixing some key not found exceptions (#231) 2020-06-19 15:19:07 -06:00
Dmitriy Ryajov 5b28e8c488
Cleanup lpstream, Connection and BufferStream (#228) 
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* don't use delegation in connection

* move connection out to own file

* don't breakout on reset

* make sure to call close on secured conn

* add lpstream tracing

* don't breackdown by conn id

* fix import

* remove unused lable

* reset  connection on exception

* add additional metrics for skipped messages

* check for nil in secure.close
 2020-06-19 11:29:43 -06:00
Dmitriy Ryajov 719744f46a
Small fixes (#230) 
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* make sure sending doesn't fail

* add `contains`

* review comment from prev pr
 2020-06-19 11:29:25 -06:00
Dmitriy Ryajov 5f75e5adc6
don't use `switch.dial` in `subscribeToPeer` (#227) 
* count published messages

* don't call `switch.dial` in `subscribeToPeer`
 2020-06-17 20:12:32 -06:00
Dmitriy Ryajov fe828d87d8
count published messages (#224) 2020-06-16 22:14:02 -06:00
Giovanni Petrantoni 224c2613e3
Remove noise padding payload (spec removed it) (#222) 2020-06-16 17:34:02 -06:00
Jacek Sieka 523eed65ce
metrics: use eth2 name for peers metric (#223) 2020-06-16 14:43:45 -06:00
Giovanni Petrantoni 0d81acafa8 Fix multistream select trace proto logging 2020-06-16 12:02:47 +09:00
Dmitriy Ryajov 9d9f793b4f
add metrics for sent messages by topic and peer (#220) 2020-06-15 17:39:03 -06:00
Dmitriy Ryajov 3c4c10d871
con't crash on nil ptr 2020-06-15 13:37:25 -06:00
Dmitriy Ryajov 7cb6c81159
Don't modify iterables while iterating them (#219) 
* don't modify iterables while iterating

* assert handlers to properly close connections
 2020-06-15 12:30:09 -06:00
Dmitriy Ryajov 9cd47fe816
add a logScope to make tracing less ugly (#218) 
* add a logScope to make tracing less ugly

* don't crash on nil pointer
 2020-06-15 12:10:41 -06:00
Dmitriy Ryajov 85b56d0b3a
Observedaddr (#217) 
* send correct observerAddr

* run tests with info log level

* set observedaddr for channels
 2020-06-12 19:56:17 -06:00
Oskar Thorén f97e2deec7
Make methods in FloodSub, GossipSub public (#216) 
Similar to https://github.com/status-im/nim-libp2p/pull/193 but doing it for all
methods to avoid this issue in PubSub in the future. Got hit by this behavior
again in rpcHandler and took me a while to figure out.

See https://github.com/oskarth/nim-private-method-skipping-case/ for minimal
repro
 2020-06-12 17:54:12 -06:00
Dmitriy Ryajov ac04ca6e31
make sure keys exist and more metrics (#215) 2020-06-11 20:20:58 -06:00
Dmitriy Ryajov 55a294a5c9
better pubsub metrics (#214) 2020-06-11 12:09:34 -06:00
Dmitriy Ryajov 6b196ad7b4
remove pubsub peer on disconnect (#212) 
* remove pubsub peer on disconnect

* make sure lock is aquired

* add $

* count upgrades/dials/disconnects
 2020-06-11 08:45:59 -06:00
Jacek Sieka 92579435b6
secio then noise (#213) 
* secio then noise

much fewer peers on witti with noise first

* comment
 2020-06-11 08:38:47 +02:00
Viktor Kirilov 1afec627c2
proper name for topics so that we can filter dynamically using chronicles (#210) 
* proper name for topics so that we can filter dynamically using chronicles

* lowercase
 2020-06-10 10:48:01 +02:00
Jacek Sieka 8d9e231a74
grab agentversion/protoversion (#211) 2020-06-09 12:42:52 -06:00
Dmitriy Ryajov 35ff99829e
close streams (#208) 
* close streams

* don't warn on missing proto
 2020-06-08 17:42:27 -06:00
Dmitriy Ryajov ee281310c0
move trace log 2020-06-08 10:40:08 -06:00
Giovanni Petrantoni 82b4ed8f44 use declareCounter rather then gauge for certain metrics 2020-06-07 16:41:23 +09:00
Giovanni Petrantoni a6a2a81711
Start adding some metrics to pubsub (#192) 
* Start adding some metrics to pubsub

In order to visualize it's functionality
Still WIP

* more metrics

* add per topic metrics

* finishup with requested metrics

* add a metrisServer define to start local server

* PR fixes and cleanup
 2020-06-07 09:15:21 +02:00
Dmitriy Ryajov 130c64f33a
don't return nil in dial (#205) 
* dont return nil in dial

* don't crash on pubsub send
 2020-06-05 18:17:05 -06:00
Zahary Karadjov 2aebae56c0
Don't rely on the side-effects from doAssert 2020-06-05 19:12:10 +03:00
Zahary Karadjov 828a80ec8f
Make the MultiAddress.init function used in NBC non-failing 2020-06-05 17:51:22 +03:00
Dmitriy Ryajov 5960d42c50
remove casts from (#203) 2020-06-02 20:21:11 -06:00
Dmitriy Ryajov bb8bff2195
add sparse message propagation tests to gossipsub (#202) 
* add sparce tests to gossipsub

* add send hooks

* remove `all`
 2020-06-02 17:53:38 -06:00
Dmitriy Ryajov 285884c20c
Close peers (#201) 
* wip

* exceptions and resource cleanup

* correct peerlifetime on disconnect

* emulate defered

* remove comment
 2020-06-02 11:32:42 -06:00
Dmitriy Ryajov c7298f34f4 additional comments 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 71640da8f2 remove close from read/write methods 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 1b4876d26d emulate `defered` 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov abf659a01a more consistent dialing proto selecting logic 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 86e1c8169c decorate observers hooks with {.raises: [Defect].} 
move hooks logic out into standalone procs

License: MIT
Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com>
 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 4df151a3a3 typos 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 27a7f8c948 move EOF flag after local close and comments 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 815282a5da remove all() 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 293b7da295 typo 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 6e0eb93d4f remove readloops 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov abc12d0fb5 move stram close to a better location 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov daef00fc7b don't crash schlesi-dev 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 640c3bdc45 better exception handling and resource cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 7ff76d76b6 better exceptions 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 95774b2b81 better cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 9c2f31262e wip: try handling child stream exceptions 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov e3f8f53620 initStream method and better exceptions handling 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov d3b79b002e better exceptions and don't fail writes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 74f8b5b5f1 resource cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 93e5805c01 better exception handling 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov d83ce4c932 exceptions and resource cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov d4bdb42046 gossipsub fixes 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 9d3cc9647b fix merge 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 3d9c0bffba read from stream 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 048b1db1ad revert back allread 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 8e9716f5c3 remove on transport close cleanup 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov cf76edb0dd make noise work again 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 5158d96eaf close connection on chronos close 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 46daed9a38 wip 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 0f691cbafd add eof and closed handling 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 62da2a05c3 wip: rework with proper half-closed 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 98117a3068 call write until all is written out 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov 802299e69a breakout from publish loop 2020-06-02 09:10:27 -06:00
Jacek Sieka 7e31210455
cover missing case in MultiAddress.init (#198) 
* cover missing case in MultiAddress.init

* raise assert on marker in protocol

* unify inits for markers / non-markers

* fix string
 2020-06-01 14:41:45 +02:00
Jacek Sieka 88dbeebf17
add side effect annotations (#197) 2020-06-01 09:25:16 +02:00
Giovanni Petrantoni 37b98ad45c
Secure managers are now sorted, giving priority to noise (#191) 
* Secure managers are now sorted, giving priority to noise

* fix nimble test command

* Fix native tests

* fix directchat sample

* Could not write to connection - reduce verbosity

* fix interop testing

* Remove more tables

* test interop fixes

* directchat fix

* fix interop/remove some deprecation
 2020-06-01 08:41:32 +02:00
Giovanni Petrantoni 6affcda937
Less exceptions more results (#188) 
* Less exceptions more results

* Fix daemonapi and interop tests

* Add multibase

* wip multiaddress

* fix the build, consuming new result types

* fix standard setup

* Simplify match, rename into MaError, add more exaustive err text

* Fix the CI issues

* Fix directchat build

* daemon api fixes

* better err messages formatting

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
 2020-05-31 16:22:49 +02:00
Giovanni Petrantoni 7c9e5c2f7a "Could not write to connection" message split between trace and debug due to log size 2020-05-30 23:47:56 +09:00
Oskar Thorén b88bfc05f8
Make GossipSub initPubSub method public (#193) 
This means we can use it from other protocols that inherit GossipSub. Otherwise,
a lot of internal state (heartbeat lock etc) doesn't get initialized properly.
 2020-05-29 09:35:03 -06:00
Dmitriy Ryajov 7b6e1c0688
Gossipsub interop (#189) 
* interop fixes

* add custom messageid provider and fix seqno

* use ECDSA for speed

* adding messageid tests

* breakout from publish loop

* addressing review comments

* remove unneded var

* dont stop broadcasting on failed peers
 2020-05-27 12:33:49 -06:00
Giovanni Petrantoni 536555138c fix "future" typo 2020-05-26 16:08:09 +09:00
Giovanni Petrantoni 4447d97234 add back forgotten message in tryAndWarn 2020-05-26 15:21:59 +09:00
Dmitriy Ryajov 9132f16927
gossipsub fixes (#186) 2020-05-21 14:24:20 -06:00
Dmitriy Ryajov ba53c08b3c
Track incoming connections (#181) 
* call write until all is written out

* wip: rework with proper half-closed

* add eof and closed handling

* wip

* close connection on chronos close

* don't use read

* make noise work again

* don't reraise just yet

* fixes after backporting

* remove on transport close cleanup

* revert back allread

* rust interop fixes

* read from stream

* inc count before closing

* rebasing master

* store incomming connections

* fix merge

* remove unneeded changes

* use internal close flag to indicate disposal
 2020-05-21 11:33:48 -06:00
Dmitriy Ryajov 7900fd9f61
Half closed (#174) 
* call write until all is written out

* add comments to lpchannel fields

* add an eof flag to signal which end closed

* wip: rework with proper half-closed

* add eof and closed handling

* propagate closes to piped

* call parent close

* moving bufferstream trackers out

* move writeLock to bufferstream

* move writeLock out

* remove unused call

* wip

* rebasing master

* fix mplex tests

* wip

* fix bufferstream after backport

* wip

* rename to differentiate from chronos tracker

* close connection on chronos close

* make reset request asyncCheck

* fix channel cleanup

* misc

* don't use read

* fix backports

* make noise work again

* proper exception handling

* don't reraise just yet

* add convenience templates

* dont double wrap

* use async pragma

* fixes after backporting

* muxer owns connection

* remove on transport close cleanup

* revert back allread

* adding some todos

* read from stream

* inc count before closing

* rebasing master

* rebase master

* use correct exception type

* use try/finally insted of defer

* fix compile in trace mode

* reset channels on mplex close
 2020-05-19 18:14:15 -06:00
Dmitriy Ryajov 681991ae48
reduce buffer size in lpchannel (#184) 2020-05-19 16:15:36 -06:00
Giovanni Petrantoni 01339c991f
Don't use and expose directly secp types (#183) 
* Don't use and expose directly secp types

* Reuse same secp type names
 2020-05-19 14:48:55 +02:00
Giovanni Petrantoni c219100e64
Use results and no exceptions in chacha and curve25519 (#182) 2020-05-19 10:22:49 +02:00
Dmitriy Ryajov f8029e7359
use sha256 digest as cache keys (#135) 
* use sha256 digest as cache keys

* rebasing master
 2020-05-18 14:49:49 -06:00
Dmitriy Ryajov 9cf1fd0216
remove generic constructor and expose serverflags (#176) 
* remove generic constructor and expose serverflags

* fix transport constructor

* fix merge issues
 2020-05-18 13:04:05 -06:00
Dmitriy Ryajov 773b738c12
don't track Connection, track StreamTransport (#177) 
* don't track Connection, track StreamTransport

* make tests more deterministic
 2020-05-18 11:05:34 -06:00
Dmitriy Ryajov 5583168965
Track all connections (#175) 
* adding to do for future refactor

* no mix conn and transport stopping or it will race

* don't use intermediary vars
 2020-05-18 09:08:10 -06:00
Dmitriy Ryajov 1819502fb5
Cleanup - tests and logging (#178) 
* make async for proper exception handling

* tryAndWarn msg messes up Exception msg

* misc: comment out tracker dumps

* cleanup mplex tests

* more informative errors

* give CI time to run

* revert change, bacause it causes races
 2020-05-18 07:49:49 -06:00
Giovanni Petrantoni 7dcb807f64
Crypto utilities resultification (#150) 2020-05-18 07:25:55 +02:00
Dmitriy Ryajov 167f42ed45
Remove read (#171) 
* use readExactly

* remove `read`

* remove read

* no more `read`
 2020-05-14 22:02:05 -06:00
Jacek Sieka 69abf5097d
handle a few exceptions (#170) 
* handle a few exceptions

Some of these are maybe too aggressive, but in return, they'll log
their exception - more refactoring needed to sort this out - right now
we get crashes on unhandled exceptions of unknown origin

* during connection setup
* while closing channels
* while processing pubsubs

* catch exceptions that are raised and don't try to catch exceptions that are not raised

* propagate cancellederror

* one more

* more

* more

* make interop tests less fragile

* Raise expiration time in gossipsub fanout test for slow CI

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
 2020-05-14 21:56:56 -06:00
Giovanni Petrantoni 005e088405
Properly track and close mplex handlers (#166) 
* Properly track and close mplex handlers

* Avoid verbose warnings

* Fix tryAndWarn trace issue

* Handle LPEOF in lpchannel close
 2020-05-12 14:45:32 +02:00
Jacek Sieka 618e01eba3
don't crash on double peerinfo close (#167) 2020-05-11 21:05:24 +02:00
Jacek Sieka 3053f03814 fix varint issues 
* fixes #111
 2020-05-11 09:12:23 -06:00
Jacek Sieka 87e1f3c61f
fix trace logging 2020-05-09 13:56:07 +02:00
Jacek Sieka fbf640794b
move channel size constant 2020-05-09 08:48:26 +02:00
Dmitriy Ryajov 6196d56fc2
check for nil observers 2020-05-08 15:44:18 -06:00
Jacek Sieka ccd019b328
use stream directly in chronosstream (#163) 
* use stream directly in chronosstream

for now, chronos.AsyncStream is not used to provide any features on top
of chronos.Stream, so in order to simplify the code, chronosstream can
be used directly.

In particular, the exception handling is broken in the current
chronosstream - opening and closing the stream is simplified this way as
well.

A future implementation that actually takes advantage of the AsyncStream
features would wrap AsyncStream instead as a separate lpstream
implementation, leaving this one as-is.

* work around chronos exception type issue
 2020-05-08 22:10:06 +02:00
Giovanni Petrantoni c889224012 Add PubSub observer+ hooks (they can modify as well) 2020-05-08 13:31:52 -06:00
Ștefan Talpalaru 268253ea18 remove Chronos type from public API 2020-05-08 13:23:36 -06:00
Ștefan Talpalaru 313f9b0952 use a Transport.serverFlags attribute 2020-05-08 13:23:36 -06:00
Ștefan Talpalaru c480e65055 use set[ServerFlags] params instead of hardcoded flags 2020-05-08 13:23:36 -06:00
Ștefan Talpalaru 125843af7d TcpTransport.listen(): enable SO_REUSEADDR 
in order to avoid failures when restarting the process while the OS is
keeping open sockets from previous runs
 2020-05-08 13:23:36 -06:00
Jacek Sieka 1efada474c
remove readLoop in secure protocols (#162) 
* remove readLoop in secure protocols, fix security issues

* fix Defect on remote sending 0-byte noise/secio message
* remove msglen from `write` (unused)
* simplify SecureConn data flow
* document some control-flow issues

* unify exception behaviour across noise and secio

* secio would not raise on mac/decryption errors

* fix compile error
 2020-05-07 14:37:46 -06:00
Jacek Sieka 330da51819
removals (#159) 
* remove unused stream methods
* reimplement some of them with proc's
* remove broken tests
* Error->Defect for defect
* warning fixes
 2020-05-06 18:31:47 +02:00
Dmitriy Ryajov 6da4d2af48
Pubsub signatures flags (#161) 
* add verify signature flag

* add sign flag to enable/disable msg signing

* moving internal tests out to their own file

* cleanup nimble file

* remove unneeded tests

* move pubsub tests out

* fix tests
 2020-05-06 11:26:08 +02:00
Jacek Sieka e1928456a7 avoid newlines in $ 
they mess with debug prints and logging (same reason why $(seq) doesn't
print them
 2020-04-28 10:59:53 -06:00
cheatfate 290ba712e9
Fix MultiAddress.protoAddress() bug for fixed arrays. 
Add tests for it.
 2020-04-28 14:43:44 +03:00
Giovanni Petrantoni 8a22c073c7 Fix secure/noise securing explicitly, added noise to pubsub tests 2020-04-24 14:33:08 -06:00
cheatfate 917b5f5c84 Add MultiAddress.init(integer) for tcp,udp,dccp,sctp protocols. 
Add tests for it.
 2020-04-23 08:10:17 -06:00
Giovanni Petrantoni 1c4d72f5e3
Use Result construct in minasn1 (#144) 2020-04-23 14:10:20 +02:00
Giovanni Petrantoni 6ae92eb21a Hotfix noise read usage, replace with readExactly 2020-04-22 23:36:59 +09:00
Giovanni Petrantoni d96372f820 Minor tweaking in errors.nim in order to give priority to Defect 2020-04-21 22:10:47 +09:00
Giovanni Petrantoni 4c6a123d31
Add chronos trackers and used them to sanitize resource disposal (#131) 
* Add chronos trackers and used them to sanitize resource disposal

* Chronos trackers for transport tests wip

* No more chronos leaks in testtransport

* Make tcp transport and test more robust when closing

* Test async leaking tracking wip

* Fix a regression in wire connect

* Add chronos trackers to more tests and sanitize resource closure

* Wip fixing floodsub tests

* Floodsub wip

* Made floodsub basically deterministic, hit a nim bug with captures tho

* Wrap up floodsub tests refactor

* Wrapping up

* Add allFuturesThrowing utility

* Fix missing allFuturesThrowing in noise tests!

* Make tests green

* attempt fixing gossipsub failing cases

* Make sure to check also fanout in waitSub

* More verbose traces

* Gossipsub test improvments

* Refactor TcpTransport remove asyncCheck

* Add Connection trackers

* Add stricter connection tracking, wip mplex fix

* More asynccheck removal, in order to avoid connection leaks

* bump chronicles requirement

* Enable tracker dump to check CI output

* Wait for more futures in testmplex

* Remove tracker dump messages

* add tryAndWarn utility, fix mplex issue with go interop

* All allFuturesThrowing to directchat too

* make sure to cleanup on transport close
 2020-04-21 10:24:42 +09:00
Jacek Sieka e8b33c64fa
secp: use upstream secp convenience api (#141) 
* secp: use upstream secp convenience api
 2020-04-17 12:51:13 +02:00
Ștefan Talpalaru eaa73ae6e8
add stream metrics (#136) 
* add stream metrics

- just BufferStream and Connection are tracked, for now
- flag checking is enforced more strictly in close(), since it became
  clear that instances are closed multiple times

* add "metrics" dependency

and sort the list
 2020-04-14 15:27:07 +02:00
Ștefan Talpalaru 7723403b1f
debug prints (#132) 
* debug prints

* CI: enable stack traces

* Azure: better NimBinaries cache key

* CI changes

- Azure: remove Linux target
- Travis: add ARM64 target

* uglify the code in order to save 12 bytes per LPStream object
 2020-04-14 15:21:16 +02:00
Jacek Sieka 2b823bde68
secp: update (#138) 2020-04-12 19:03:08 +02:00
Giovanni Petrantoni 303ec297da
Start removing allFutures (#125) 
* Start removing allFutures

* More allfutures removal

* Complete allFutures removal except legacy and tests

* Introduce table values copies to prevent error

* Switch to allFinished

* Resolve TODOs in flood/gossip

* muxer handler, log and re-raise

* Add a common and flexible way to check multiple futures
 2020-04-11 13:08:25 +09:00
Dmitriy Ryajov f4740c8b8e fix trace runs in connection 2020-04-07 14:55:05 -06:00
Dmitriy Ryajov 00fbc9246e fix nil condition 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 6cbcc7859e reduse usssage of asyncCheck 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov bd49a35e0a formatting 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 976164ba3c proper connection cleanup 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 0c8dde15e7 remove unused field 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 4ee1b4a66d connection closing tests 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 5a657c9264 disable storing connections on internal table 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 87b35de83e close connections 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov bae4713a85 small cleanup 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov 8dda6c289d more connection closes to fix leaks 2020-04-07 12:16:59 -06:00
Dmitriy Ryajov e4303110a6 properly close connections 2020-04-07 12:16:59 -06:00
Giovanni Petrantoni 35a48fa560 Re-enable gossipsub internal tests when running CI 
minor bonus: add a link in the comments about bearssl issue with
callbacks
 2020-04-07 22:07:00 +09:00
Giovanni Petrantoni a576069305 Chachapoly new bearssl fixes 2020-04-07 17:52:19 +09:00
Giovanni Petrantoni f7af2c2e5c Fix possible crash in tcptransport when server is nil 2020-04-06 12:56:17 +09:00
Giovanni Petrantoni 3514733060 Fix table assertion, edited while iterating 
(the fix is not so nice.. adds plenty of allocations, but for now should
be ok)
 2020-04-05 01:19:10 +09:00
Giovanni Petrantoni e39bf0a4cf
Mplex: Add the ability to send any size payload (#123) 
* Mplex: Add the ability to send any size payload

* Ensure size of coder header
 2020-04-03 17:26:46 +02:00
Ștefan Talpalaru 7f8090b166
SecioConn.readMessage(): fix the exception types 2020-04-02 19:07:14 +02:00
Dmitriy Ryajov 5285f0d091
Fix/misc (#116) 
* only check for payload size

* only subscribe if connection succeeded

* fix failing test

* check that the strem is active before openning

* msg type should not be > than 0x7

* fix tests

* check max against enum val
 2020-03-29 08:28:48 -06:00
Dmitriy Ryajov a35ecc6227 reverting connection changes 2020-03-28 09:29:43 -06:00
Dmitriy Ryajov 3effb95f10 close underlying bufferstream in lpchannel 2020-03-28 09:29:43 -06:00
Giovanni Petrantoni 56392913f9
Allow to specify a read size in readLp (#117) 
* Allow to specify a read size in readLp

Also fix secio fragments + test

* Revert readLp changes, use just `read` in test
 2020-03-26 15:06:47 +09:00
Giovanni Petrantoni 33ceefb209 Remove nim random usage from noise 2020-03-26 15:04:46 +09:00
Giovanni Petrantoni b132eae939 Add secio oversized payload fragmentation 
Cannot be used because Connection Read size hard limit tho
 2020-03-25 16:53:35 +09:00
Giovanni Petrantoni 1550bea1cc Support arbitrary stream write sizes in noise 2020-03-25 16:10:11 +09:00
Dmitriy Ryajov 4199508f17 readMplexVarint should return uint64 2020-03-24 15:41:40 -06:00
Dmitriy Ryajov 2de98751ae fix: use exact types for mplex id 2020-03-24 15:41:40 -06:00
cheatfate 1f5d994700
Fix compilation errors introduced by latest chronos. 2020-03-24 09:48:05 +02:00
Giovanni Petrantoni 3d10513b3e Properly await the handshake end in secure handler 2020-03-24 15:34:02 +09:00
Giovanni Petrantoni 0a3e4a764b
Less verbose traces (#112) 
* Make traces less verbose with shortHexDump utility

* Rename shortHexDump into shortLog

* Improve shortLog, add shortLog for crypto keys

* Add proper shortLog implementations in messages
 2020-03-23 15:03:36 +09:00
Giovanni Petrantoni b1a34f478e Avoid 2 async consecutive writes in noise HS 2020-03-18 20:40:51 +09:00
Giovanni Petrantoni c02fca25f8
Noise (#90) 
* Start ChaCha20Poly1305 integration (BearSSL)

* Add Curve25519 (BearSSL) required operations for noise

* Fix curve mulgen iterate/derive

* Fix misleading header

* Add chachapoly proper test

* Curve25519 integration tests (failing, something is wrong)

* Add few converters, finish c25519 integration tests

* Remove implicit converters

* removed internal globals

* Start noise implementation

* Fix public() using proper bear mulgen

* Noise protocol WIP

* Noise progress

* Add a quick nim version of HKDF

* Converted hkdf to iterator, useful for noise

* Noise protocol implementation progress

* Noise progress

* XX handshake almost there

* noise progress

* Noise, testing handshake with test vectors

* Noise handshake progress, still wrong somewhere!

* Noise handshake success!

* Full verified noise XX handshake completed

* Fix and rewrite test to be similar to switch one

* Start with connection upgrade

* Switch chachapoly to CT implementations

* Improve HKDF implementation

* Use a type insted of tuple for HandshakeResult

* Remove unnecessary Let

* More cosmetic fixes

* Properly check randomBytes result

* Fix chachapoly signature

* Noise full circle (altho dispatcher is nil cursed)

* Allow nil aads in chachapoly routines

* Noise implementation up to running full test

* Use bearssl HKDF as well

* Directly use bearssl rng for curve25519 keys

* Add a (disabled/no CI) noise interop test server

* WIP on fixing interop issues

* More fixes in noise implementation for interop

* bump chronos requirement (nimble)

* Add a chachapoly test for very small size payloads

* Noise, more tracing

* Add 2 properly working noise tests

* Fix payload packing, following the spec properly (and not go version but
rather rust)

* Sanity, replace discard with asyncCheck

* Small fixes and optimization

* Use stew endian2 rather then system endian module

* Update nimble deps (chronos)

* Minor cosmetic/code sanity fixes

* Noise, handle Nonce max

* Noise tests, make sure to close secured conns

* More polish, improve code readability too

* More polish and testing again which test fails

* Further polishing

* Restore noise tests

* Remove useless Future[void]

* Remove useless CipherState initializer

* add a proper read wait future in second noise test

* Remove noise generic secure implementation for now

* Few fixes to run eth2 sim

* Add more debug info in noise traces

* Merge size + payload write in sendEncryptedMessage

* Revert secure interface, add outgoing property directly in newNoise

* remove sendEncrypted and receiveEncrypted

* Use openarray in chachapoly and curve25519 helpers
 2020-03-17 13:30:01 +01:00
Dmitriy Ryajov 761b221b75 don't trace the payload 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 5c234f704d added 1 byte jitter test 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 406b79887d add jitter tolerance to mplex 2020-03-12 08:01:43 -06:00
Dmitriy Ryajov 82af623641 use correct shift 2020-03-11 09:58:24 -06:00
Dmitriy Ryajov 07695b2260 use `readExactly` not `read` to read all bytes 2020-03-11 09:58:24 -06:00
Jacek Sieka 52cb7ced04
prefer endians2 (#98) 2020-03-10 21:40:52 +01:00
Dmitriy Ryajov df29ac760e remove missing include 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov fbe54e5c5a move SecureConn to secure.nim 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov 19095a0f85 move `secure` to base as well 2020-03-09 13:26:38 -06:00
Eugene Kabanov d1c6591b8a Fix and refactoring of some procedures which are able to return nil as result (#97) 
* Fix do not return nil as result.

* Fix mplex test to properly raise.
 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov c3e791db0d remove unused imports 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov f1a297f0f3 add initiator flag to handshake 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov d1d133319e change handleConn to be a proc 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov 9aa9e97602 feat: move connection handling logic to base class 2020-03-09 13:26:38 -06:00
Dmitriy Ryajov 871d8d7478 remove white space 2020-03-09 13:26:38 -06:00
Eugene Kabanov 5701d937c8
Signed variable integers fixes. (#96) 
* Fix signed varints.
Add tests for signed varints.
Remove some casts to allow usage at compile time.

* Fix vsizeof() on 32bit platforms.

* Add `hint` and `zint` types for proper signed integer encoding.

* Fix varint related bugs.

* Update requirements.

* Fix interop tests because of fixed readLine.

* Add putVarint, getVarint and tests.
 2020-03-06 20:19:43 +01:00
Eugene Kabanov 381630f185
Fix and refactoring of some procedures which are able to return nil as result (#97) 
* Fix do not return nil as result.

* Fix mplex test to properly raise.
 2020-03-04 21:45:14 +02:00
Dmitriy Ryajov ffc3b04222 read should await for data on an empty buff 2020-03-01 04:06:42 -06:00
Eugene Kabanov 91d2f52f4e
Add some utility procedures to MultiAddress. (#94) 
* Add some utility procedures to MultiAddress.

* Do not use result and add more operations to be more effective for compiler.
 2020-02-27 19:26:42 +02:00
Dmitriy Ryajov 85d4baca95 await connection 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 946b9580c0 don't use nil result 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 52c270dcf9 warn on failed pubsub subscribe 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov eb49d4b218 no empty proto dials and add connect method 2020-02-25 17:52:08 -06:00
Dmitriy Ryajov 9efc08cb2f fix: #83 2020-02-24 08:58:02 -06:00
Dmitriy Ryajov 6a7f9f058c address review comments 2020-02-21 09:21:06 -06:00
Dmitriy Ryajov fbcef69891 implicitelly dial pubsub if enabled 2020-02-21 09:21:06 -06:00
Giovanni Petrantoni cb156f3260 Fix public() using proper bear mulgen 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 93a480e6f3 removed internal globals 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni e0e371994c Remove implicit converters 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni f039853a19 Add few converters, finish c25519 integration tests 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 99a169b65d Curve25519 integration tests (failing, something is wrong) 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 2103aa59f3 Add chachapoly proper test 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 6ba274de27 Fix misleading header 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni f47fb628f0 Fix curve mulgen iterate/derive 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 8dec368878 Add Curve25519 (BearSSL) required operations for noise 2020-02-18 09:35:14 -06:00
Giovanni Petrantoni 71998ffc67 Start ChaCha20Poly1305 integration (BearSSL) 2020-02-18 09:35:14 -06:00
Dmitriy Ryajov 1a987a9c5b fix: delete mplex channel 2020-02-16 13:51:03 -06:00
Dmitriy Ryajov 65ed2c7cee make code more robust 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 9023bf786d remove sleeps 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov acdaeb8f5d working out synchronization issues 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov b1dd564c32 make sure we don't exit the handler prematurely 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f9ad113d11 don't send close message if remote closed 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f767614827 remove unnecesary async lock 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 934c858542 increase timeout to allow floodsub to finish 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 26a4510377 remove unnecesary nil check in peerinfo 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 0349165830 typo 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 7f8eb0272e cleanup and fix tests 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 6316b07e81 don't crash when stringifying nil peerinfo 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov 4faf35c52c throw exception if handshake failed 2020-02-16 11:31:35 -06:00
Dmitriy Ryajov f4e9bc8bfb remove sleepAsync for synchronization 2020-02-16 11:31:35 -06:00