Dmitriy Ryajov
6196d56fc2
check for nil observers
2020-05-08 15:44:18 -06:00
Jacek Sieka
ccd019b328
use stream directly in chronosstream ( #163 )
...
* use stream directly in chronosstream
for now, chronos.AsyncStream is not used to provide any features on top
of chronos.Stream, so in order to simplify the code, chronosstream can
be used directly.
In particular, the exception handling is broken in the current
chronosstream - opening and closing the stream is simplified this way as
well.
A future implementation that actually takes advantage of the AsyncStream
features would wrap AsyncStream instead as a separate lpstream
implementation, leaving this one as-is.
* work around chronos exception type issue
2020-05-08 22:10:06 +02:00
Giovanni Petrantoni
c889224012
Add PubSub observer+ hooks (they can modify as well)
2020-05-08 13:31:52 -06:00
Ștefan Talpalaru
268253ea18
remove Chronos type from public API
2020-05-08 13:23:36 -06:00
Ștefan Talpalaru
313f9b0952
use a Transport.serverFlags attribute
2020-05-08 13:23:36 -06:00
Ștefan Talpalaru
c480e65055
use set[ServerFlags] params instead of hardcoded flags
2020-05-08 13:23:36 -06:00
Ștefan Talpalaru
125843af7d
TcpTransport.listen(): enable SO_REUSEADDR
...
in order to avoid failures when restarting the process while the OS is
keeping open sockets from previous runs
2020-05-08 13:23:36 -06:00
Jacek Sieka
1efada474c
remove readLoop in secure protocols ( #162 )
...
* remove readLoop in secure protocols, fix security issues
* fix Defect on remote sending 0-byte noise/secio message
* remove msglen from `write` (unused)
* simplify SecureConn data flow
* document some control-flow issues
* unify exception behaviour across noise and secio
* secio would not raise on mac/decryption errors
* fix compile error
2020-05-07 14:37:46 -06:00
Jacek Sieka
330da51819
removals ( #159 )
...
* remove unused stream methods
* reimplement some of them with proc's
* remove broken tests
* Error->Defect for defect
* warning fixes
2020-05-06 18:31:47 +02:00
Dmitriy Ryajov
6da4d2af48
Pubsub signatures flags ( #161 )
...
* add verify signature flag
* add sign flag to enable/disable msg signing
* moving internal tests out to their own file
* cleanup nimble file
* remove unneeded tests
* move pubsub tests out
* fix tests
2020-05-06 11:26:08 +02:00
Jacek Sieka
e1928456a7
avoid newlines in $
...
they mess with debug prints and logging (same reason why $(seq) doesn't
print them
2020-04-28 10:59:53 -06:00
cheatfate
290ba712e9
Fix MultiAddress.protoAddress() bug for fixed arrays.
...
Add tests for it.
2020-04-28 14:43:44 +03:00
Giovanni Petrantoni
8a22c073c7
Fix secure/noise securing explicitly, added noise to pubsub tests
2020-04-24 14:33:08 -06:00
cheatfate
917b5f5c84
Add MultiAddress.init(integer) for tcp,udp,dccp,sctp protocols.
...
Add tests for it.
2020-04-23 08:10:17 -06:00
Giovanni Petrantoni
1c4d72f5e3
Use Result construct in minasn1 ( #144 )
2020-04-23 14:10:20 +02:00
Giovanni Petrantoni
6ae92eb21a
Hotfix noise read usage, replace with readExactly
2020-04-22 23:36:59 +09:00
Giovanni Petrantoni
d96372f820
Minor tweaking in errors.nim in order to give priority to Defect
2020-04-21 22:10:47 +09:00
Giovanni Petrantoni
4c6a123d31
Add chronos trackers and used them to sanitize resource disposal ( #131 )
...
* Add chronos trackers and used them to sanitize resource disposal
* Chronos trackers for transport tests wip
* No more chronos leaks in testtransport
* Make tcp transport and test more robust when closing
* Test async leaking tracking wip
* Fix a regression in wire connect
* Add chronos trackers to more tests and sanitize resource closure
* Wip fixing floodsub tests
* Floodsub wip
* Made floodsub basically deterministic, hit a nim bug with captures tho
* Wrap up floodsub tests refactor
* Wrapping up
* Add allFuturesThrowing utility
* Fix missing allFuturesThrowing in noise tests!
* Make tests green
* attempt fixing gossipsub failing cases
* Make sure to check also fanout in waitSub
* More verbose traces
* Gossipsub test improvments
* Refactor TcpTransport remove asyncCheck
* Add Connection trackers
* Add stricter connection tracking, wip mplex fix
* More asynccheck removal, in order to avoid connection leaks
* bump chronicles requirement
* Enable tracker dump to check CI output
* Wait for more futures in testmplex
* Remove tracker dump messages
* add tryAndWarn utility, fix mplex issue with go interop
* All allFuturesThrowing to directchat too
* make sure to cleanup on transport close
2020-04-21 10:24:42 +09:00
Jacek Sieka
e8b33c64fa
secp: use upstream secp convenience api ( #141 )
...
* secp: use upstream secp convenience api
2020-04-17 12:51:13 +02:00
Ștefan Talpalaru
eaa73ae6e8
add stream metrics ( #136 )
...
* add stream metrics
- just BufferStream and Connection are tracked, for now
- flag checking is enforced more strictly in close(), since it became
clear that instances are closed multiple times
* add "metrics" dependency
and sort the list
2020-04-14 15:27:07 +02:00
Ștefan Talpalaru
7723403b1f
debug prints ( #132 )
...
* debug prints
* CI: enable stack traces
* Azure: better NimBinaries cache key
* CI changes
- Azure: remove Linux target
- Travis: add ARM64 target
* uglify the code in order to save 12 bytes per LPStream object
2020-04-14 15:21:16 +02:00
Jacek Sieka
2b823bde68
secp: update ( #138 )
2020-04-12 19:03:08 +02:00
Giovanni Petrantoni
303ec297da
Start removing allFutures ( #125 )
...
* Start removing allFutures
* More allfutures removal
* Complete allFutures removal except legacy and tests
* Introduce table values copies to prevent error
* Switch to allFinished
* Resolve TODOs in flood/gossip
* muxer handler, log and re-raise
* Add a common and flexible way to check multiple futures
2020-04-11 13:08:25 +09:00
Dmitriy Ryajov
f4740c8b8e
fix trace runs in connection
2020-04-07 14:55:05 -06:00
Dmitriy Ryajov
00fbc9246e
fix nil condition
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
6cbcc7859e
reduse usssage of asyncCheck
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
bd49a35e0a
formatting
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
976164ba3c
proper connection cleanup
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
0c8dde15e7
remove unused field
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
4ee1b4a66d
connection closing tests
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
5a657c9264
disable storing connections on internal table
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
87b35de83e
close connections
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
bae4713a85
small cleanup
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
8dda6c289d
more connection closes to fix leaks
2020-04-07 12:16:59 -06:00
Dmitriy Ryajov
e4303110a6
properly close connections
2020-04-07 12:16:59 -06:00
Giovanni Petrantoni
35a48fa560
Re-enable gossipsub internal tests when running CI
...
minor bonus: add a link in the comments about bearssl issue with
callbacks
2020-04-07 22:07:00 +09:00
Giovanni Petrantoni
a576069305
Chachapoly new bearssl fixes
2020-04-07 17:52:19 +09:00
Giovanni Petrantoni
f7af2c2e5c
Fix possible crash in tcptransport when server is nil
2020-04-06 12:56:17 +09:00
Giovanni Petrantoni
3514733060
Fix table assertion, edited while iterating
...
(the fix is not so nice.. adds plenty of allocations, but for now should
be ok)
2020-04-05 01:19:10 +09:00
Giovanni Petrantoni
e39bf0a4cf
Mplex: Add the ability to send any size payload ( #123 )
...
* Mplex: Add the ability to send any size payload
* Ensure size of coder header
2020-04-03 17:26:46 +02:00
Ștefan Talpalaru
7f8090b166
SecioConn.readMessage(): fix the exception types
2020-04-02 19:07:14 +02:00
Dmitriy Ryajov
5285f0d091
Fix/misc ( #116 )
...
* only check for payload size
* only subscribe if connection succeeded
* fix failing test
* check that the strem is active before openning
* msg type should not be > than 0x7
* fix tests
* check max against enum val
2020-03-29 08:28:48 -06:00
Dmitriy Ryajov
a35ecc6227
reverting connection changes
2020-03-28 09:29:43 -06:00
Dmitriy Ryajov
3effb95f10
close underlying bufferstream in lpchannel
2020-03-28 09:29:43 -06:00
Giovanni Petrantoni
56392913f9
Allow to specify a read size in readLp ( #117 )
...
* Allow to specify a read size in readLp
Also fix secio fragments + test
* Revert readLp changes, use just `read` in test
2020-03-26 15:06:47 +09:00
Giovanni Petrantoni
33ceefb209
Remove nim random usage from noise
2020-03-26 15:04:46 +09:00
Giovanni Petrantoni
b132eae939
Add secio oversized payload fragmentation
...
Cannot be used because Connection Read size hard limit tho
2020-03-25 16:53:35 +09:00
Giovanni Petrantoni
1550bea1cc
Support arbitrary stream write sizes in noise
2020-03-25 16:10:11 +09:00
Dmitriy Ryajov
4199508f17
readMplexVarint should return uint64
2020-03-24 15:41:40 -06:00
Dmitriy Ryajov
2de98751ae
fix: use exact types for mplex id
2020-03-24 15:41:40 -06:00
cheatfate
1f5d994700
Fix compilation errors introduced by latest chronos.
2020-03-24 09:48:05 +02:00
Giovanni Petrantoni
3d10513b3e
Properly await the handshake end in secure handler
2020-03-24 15:34:02 +09:00
Giovanni Petrantoni
0a3e4a764b
Less verbose traces ( #112 )
...
* Make traces less verbose with shortHexDump utility
* Rename shortHexDump into shortLog
* Improve shortLog, add shortLog for crypto keys
* Add proper shortLog implementations in messages
2020-03-23 15:03:36 +09:00
Giovanni Petrantoni
b1a34f478e
Avoid 2 async consecutive writes in noise HS
2020-03-18 20:40:51 +09:00
Giovanni Petrantoni
c02fca25f8
Noise ( #90 )
...
* Start ChaCha20Poly1305 integration (BearSSL)
* Add Curve25519 (BearSSL) required operations for noise
* Fix curve mulgen iterate/derive
* Fix misleading header
* Add chachapoly proper test
* Curve25519 integration tests (failing, something is wrong)
* Add few converters, finish c25519 integration tests
* Remove implicit converters
* removed internal globals
* Start noise implementation
* Fix public() using proper bear mulgen
* Noise protocol WIP
* Noise progress
* Add a quick nim version of HKDF
* Converted hkdf to iterator, useful for noise
* Noise protocol implementation progress
* Noise progress
* XX handshake almost there
* noise progress
* Noise, testing handshake with test vectors
* Noise handshake progress, still wrong somewhere!
* Noise handshake success!
* Full verified noise XX handshake completed
* Fix and rewrite test to be similar to switch one
* Start with connection upgrade
* Switch chachapoly to CT implementations
* Improve HKDF implementation
* Use a type insted of tuple for HandshakeResult
* Remove unnecessary Let
* More cosmetic fixes
* Properly check randomBytes result
* Fix chachapoly signature
* Noise full circle (altho dispatcher is nil cursed)
* Allow nil aads in chachapoly routines
* Noise implementation up to running full test
* Use bearssl HKDF as well
* Directly use bearssl rng for curve25519 keys
* Add a (disabled/no CI) noise interop test server
* WIP on fixing interop issues
* More fixes in noise implementation for interop
* bump chronos requirement (nimble)
* Add a chachapoly test for very small size payloads
* Noise, more tracing
* Add 2 properly working noise tests
* Fix payload packing, following the spec properly (and not go version but
rather rust)
* Sanity, replace discard with asyncCheck
* Small fixes and optimization
* Use stew endian2 rather then system endian module
* Update nimble deps (chronos)
* Minor cosmetic/code sanity fixes
* Noise, handle Nonce max
* Noise tests, make sure to close secured conns
* More polish, improve code readability too
* More polish and testing again which test fails
* Further polishing
* Restore noise tests
* Remove useless Future[void]
* Remove useless CipherState initializer
* add a proper read wait future in second noise test
* Remove noise generic secure implementation for now
* Few fixes to run eth2 sim
* Add more debug info in noise traces
* Merge size + payload write in sendEncryptedMessage
* Revert secure interface, add outgoing property directly in newNoise
* remove sendEncrypted and receiveEncrypted
* Use openarray in chachapoly and curve25519 helpers
2020-03-17 13:30:01 +01:00
Dmitriy Ryajov
761b221b75
don't trace the payload
2020-03-12 08:01:43 -06:00
Dmitriy Ryajov
5c234f704d
added 1 byte jitter test
2020-03-12 08:01:43 -06:00
Dmitriy Ryajov
406b79887d
add jitter tolerance to mplex
2020-03-12 08:01:43 -06:00
Dmitriy Ryajov
82af623641
use correct shift
2020-03-11 09:58:24 -06:00
Dmitriy Ryajov
07695b2260
use `readExactly` not `read` to read all bytes
2020-03-11 09:58:24 -06:00
Jacek Sieka
52cb7ced04
prefer endians2 ( #98 )
2020-03-10 21:40:52 +01:00
Dmitriy Ryajov
df29ac760e
remove missing include
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
fbe54e5c5a
move SecureConn to secure.nim
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
19095a0f85
move `secure` to base as well
2020-03-09 13:26:38 -06:00
Eugene Kabanov
d1c6591b8a
Fix and refactoring of some procedures which are able to return nil as result ( #97 )
...
* Fix do not return nil as result.
* Fix mplex test to properly raise.
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
c3e791db0d
remove unused imports
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
f1a297f0f3
add initiator flag to handshake
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
d1d133319e
change handleConn to be a proc
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
9aa9e97602
feat: move connection handling logic to base class
2020-03-09 13:26:38 -06:00
Dmitriy Ryajov
871d8d7478
remove white space
2020-03-09 13:26:38 -06:00
Eugene Kabanov
5701d937c8
Signed variable integers fixes. ( #96 )
...
* Fix signed varints.
Add tests for signed varints.
Remove some casts to allow usage at compile time.
* Fix vsizeof() on 32bit platforms.
* Add `hint` and `zint` types for proper signed integer encoding.
* Fix varint related bugs.
* Update requirements.
* Fix interop tests because of fixed readLine.
* Add putVarint, getVarint and tests.
2020-03-06 20:19:43 +01:00
Eugene Kabanov
381630f185
Fix and refactoring of some procedures which are able to return nil as result ( #97 )
...
* Fix do not return nil as result.
* Fix mplex test to properly raise.
2020-03-04 21:45:14 +02:00
Dmitriy Ryajov
ffc3b04222
read should await for data on an empty buff
2020-03-01 04:06:42 -06:00
Eugene Kabanov
91d2f52f4e
Add some utility procedures to MultiAddress. ( #94 )
...
* Add some utility procedures to MultiAddress.
* Do not use result and add more operations to be more effective for compiler.
2020-02-27 19:26:42 +02:00
Dmitriy Ryajov
85d4baca95
await connection
2020-02-25 17:52:08 -06:00
Dmitriy Ryajov
946b9580c0
don't use nil result
2020-02-25 17:52:08 -06:00
Dmitriy Ryajov
52c270dcf9
warn on failed pubsub subscribe
2020-02-25 17:52:08 -06:00
Dmitriy Ryajov
eb49d4b218
no empty proto dials and add connect method
2020-02-25 17:52:08 -06:00
Dmitriy Ryajov
9efc08cb2f
fix : #83
2020-02-24 08:58:02 -06:00
Dmitriy Ryajov
6a7f9f058c
address review comments
2020-02-21 09:21:06 -06:00
Dmitriy Ryajov
fbcef69891
implicitelly dial pubsub if enabled
2020-02-21 09:21:06 -06:00
Giovanni Petrantoni
cb156f3260
Fix public() using proper bear mulgen
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
93a480e6f3
removed internal globals
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
e0e371994c
Remove implicit converters
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
f039853a19
Add few converters, finish c25519 integration tests
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
99a169b65d
Curve25519 integration tests (failing, something is wrong)
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
2103aa59f3
Add chachapoly proper test
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
6ba274de27
Fix misleading header
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
f47fb628f0
Fix curve mulgen iterate/derive
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
8dec368878
Add Curve25519 (BearSSL) required operations for noise
2020-02-18 09:35:14 -06:00
Giovanni Petrantoni
71998ffc67
Start ChaCha20Poly1305 integration (BearSSL)
2020-02-18 09:35:14 -06:00
Dmitriy Ryajov
1a987a9c5b
fix: delete mplex channel
2020-02-16 13:51:03 -06:00
Dmitriy Ryajov
65ed2c7cee
make code more robust
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
9023bf786d
remove sleeps
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
acdaeb8f5d
working out synchronization issues
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
b1dd564c32
make sure we don't exit the handler prematurely
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
f9ad113d11
don't send close message if remote closed
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
f767614827
remove unnecesary async lock
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
934c858542
increase timeout to allow floodsub to finish
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
26a4510377
remove unnecesary nil check in peerinfo
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
0349165830
typo
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
7f8eb0272e
cleanup and fix tests
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
6316b07e81
don't crash when stringifying nil peerinfo
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
4faf35c52c
throw exception if handshake failed
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
f4e9bc8bfb
remove sleepAsync for synchronization
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
53e163abf2
check for size bounds when reading varints
2020-02-16 11:31:35 -06:00
Dmitriy Ryajov
94fc4e6fd2
don't use sleeps for synchronization
2020-02-16 11:31:35 -06:00
Giovanni Petrantoni
39dc9ad8a3
Revert "Implement timeout in mplex in order to read reliably close state"
...
This reverts commit 314cb6e6bffe122bcfbd155a4ef830849ace66ef.
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
11bd55308f
Revert "Properly use already specified MaxReadWriteTime as timeout"
...
This reverts commit 8f5dd75e7f28da08de2bf393b93cd290b82ed78b.
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
ea0d167285
Fix max size write mplex test
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
cf5dd27e57
Properly use already specified MaxReadWriteTime as timeout
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
4034dfae19
Implement timeout in mplex in order to read reliably close state
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
136ac3f550
Use already existing error for channel push over size
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
f9cbdc252f
Improve connection.nim read error clarity when buffer overflow
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
120ba0d528
Add a safety check on message size limit when pushing new data in mplex
2020-02-12 11:57:38 -05:00
Giovanni Petrantoni
3531ebf772
Fix typo
2020-02-12 11:57:38 -05:00
cheatfate
5b6f4bf5fa
Export MultiAddress pattern procedures.
2020-02-12 12:04:12 +02:00
Eugene Kabanov
540e79a430
Add peer lifetime feature for PeerInfo. ( #77 )
...
* Add peer lifetime feature for PeerInfo.
Refactor peerinfo to use openarrays instead of sequences.
Fix tests and examples to use arrays instead of sequences.
* Add access to lifetime Future[T] itself.
2020-02-11 12:53:39 -05:00
Giovanni Petrantoni
23712ecf3b
Lazy channels ( #78 )
...
* Implemented lazy stream opening for mplex connections
* Properly fix newStream usage
* Make lazy channel open optional
* Add Lazy channel test
* Cleanup mplex test
* Move lazyness properly into LPChannel
* Connection writeLp back to proc
2020-02-11 12:30:36 -05:00
Dmitriy Ryajov
88a030d8fb
fix: removing timeouts from conn
2020-02-05 20:38:43 +01:00
Dmitriy Ryajov
2232ca598e
don't timeout in pubsub
2020-02-04 17:59:57 +01:00
Zahary Karadjov
1bd933cd5a
More precise tracing
2020-02-04 17:27:32 +01:00
Zahary Karadjov
7bd305471c
Make sure the library can compile with json logging in trace mode
2020-02-04 15:17:39 +01:00
Dmitriy Ryajov
691efaa8a1
dont use result
2020-01-30 09:30:08 -06:00
Dmitriy Ryajov
171d60fb52
fix: reraise exception in readLp
2020-01-30 09:30:08 -06:00
cheatfate
06886290dd
Add links to specifications to varint comments.
2020-01-29 11:14:25 +02:00
Eugene Kabanov
d902127595
Fix LibP2P varint implementation to follow latest specification. ( #65 )
...
* Fix LibP2P varint implementation to decode only minimal valus.
2020-01-28 16:29:42 +01:00
Dmitriy Ryajov
007b0219f3
use ECDSA as defaults, faster in tests
2020-01-09 12:55:21 -06:00
Dmitriy Ryajov
667691f784
send messages in batches
2020-01-09 12:55:21 -06:00
Dmitriy Ryajov
cd8961cfb9
close and cleanup connections
2020-01-09 12:55:21 -06:00
Dmitriy Ryajov
c6561b8851
add timeouts to connection/secio
2020-01-09 12:55:21 -06:00
cheatfate
107e71203d
Fix #61 .
2020-01-09 17:38:10 +02:00
Dmitriy Ryajov
58cdefac21
don't expose private api
2019-12-24 10:35:35 -06:00
Dmitriy Ryajov
0fb1f1c5b8
strenghten pubsub interop testing
2019-12-24 10:35:35 -06:00
Dmitriy Ryajov
8714c66353
allow empty protocol string in dial
2019-12-24 10:35:35 -06:00
Dmitriy Ryajov
68cc57669e
Feat/pubsub validators ( #58 )
...
* feat: adding validator hooks to pubsub
* expose add/remove validators on switch
* do less unnecessary copyng
2019-12-16 23:24:03 -06:00
Dmitriy Ryajov
b6b0cdea98
fix: inconsistent error handling ( #56 )
...
* fix: inconsistent error handling
* fixing a few more returns
* remov reduntant returns
* feat: dont sleep in readLoop
2019-12-13 09:25:07 -06:00
Yuriy Glukhov
e0aae6d8ac
Moved base32, 58 and 64 to stew
2019-12-13 16:00:39 +02:00
Dmitriy Ryajov
293a219dbe
Cleanup ( #55 )
...
* fix: don't allow replacing pubkey
* fix: several small improvements
* removing pubkey setter
* improove error handling
* remove the use of Option[T] if not needed
* don't use optional
* fix-ci: temporarily pin p2pd to a working tag
* fix example to comply with latest changes
* bumping p2pd again to a higher version
2019-12-10 14:50:35 -06:00
Zahary Karadjov
77e866d29a
Address review comments
2019-12-09 19:17:29 +02:00
Zahary Karadjov
454f658ba8
Fixes and tweaks related to the beacon node integration
...
* Bugfix: Dialing an already connected peer may lead to crash
* Introduced a standard_setup module allowing to instantiate
the `Switch` object in an easier manner.
* Added `Switch.disconnect(peer)`
* Trailing space removed (sorry about polluting the diff)
2019-12-08 23:58:43 +02:00
Dmitriy Ryajov
5f6fcc3d90
extract public and private keys fields from peerid ( #44 )
...
* extract public and private keys fields from peerid
* allow assigning a public key
* cleaned up TODOs
* make pubsub prefix a const
* public key should be an `Option`
2019-12-07 10:36:39 -06:00
Dmitriy Ryajov
e623e70e7b
PubSub (Gossip & Flood) Implementation ( #36 )
...
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon.
* add close event
* wip: gossipsub
* splitting rpc message
* making message handling more consistent
* initial gossipsub implementation
* feat: nim 1.0 cleanup
* wip: gossipsub protobuf
* adding encoding/decoding of gossipsub messages
* add disconnect handler
* add proper gossipsub msg handling
* misc: cleanup for nim 1.0
* splitting floodsub and gossipsub tests
* feat: add mesh rebalansing
* test pubsub
* add mesh rebalansing tests
* testing mesh maintenance
* finishing mcache implementatin
* wip: commenting out broken tests
* wip: don't run heartbeat for now
* switchout debug for trace logging
* testing gossip peer selection algorithm
* test stream piping
* more work around message amplification
* get the peerid from message
* use timed cache as backing store
* allow setting timeout in constructor
* several changes to improve performance
* more through testing of msg amplification
* prevent gc issues
* allow piping to self and prevent deadlocks
* improove floodsub
* allow running hook on cache eviction
* prevent race conditions
* prevent race conditions and improove tests
* use hashes as cache keys
* removing useless file
* don't create a new seq
* re-enable pubsub tests
* fix imports
* reduce number of runs to speed up tests
* break out control message processing
* normalize sleeps between steps
* implement proper transport filtering
* initial interop testing
* clean up floodsub publish logic
* allow dialing without a protocol
* adding multiple reads/writes
* use protobuf varint in mplex
* don't loose conn's peerInfo
* initial interop pubsub tests
* don't duplicate connections/peers
* bring back interop tests
* wip: interop
* re-enable interop and daemon tests
* add multiple read write tests from handlers
* don't cleanup channel prematurely
* use correct channel to send/receive msgs
* adjust tests with latest changes
* include interop tests
* remove temp logging output
* fix ci
* use correct public key serialization
* additional tests for pubsub interop
2019-12-05 20:16:18 -06:00
Dmitriy Ryajov
903e79ede1
Feat/conn cleanup ( #41 )
...
Backporting proper connection cleanup from #36 to align with latest chronos changes.
* add close event
* use proper varint encoding
* add proper channel cleanup in mplex
* add connection cleanup in secio
* tidy up
* add dollar operator
* fix tests
* don't close connections prematurely
* handle closing streams properly
* misc
* implement address filtering logic
* adding pipe tests
* don't use gcsafe if not needed
* misc
* proper connection cleanup and stream muxing
* re-enable pubsub tests
2019-12-03 22:44:54 -06:00
Dmitriy Ryajov
1df16bdbce
set log level to trace - not enabled by default
2019-12-02 18:43:21 -06:00
Zahary Karadjov
f9eed172d4
Don't handle Exception
2019-12-03 01:26:59 +02:00
Dmitriy Ryajov
26366f8b56
use `Exception` to emulate `getCurrentException()`
2019-12-03 01:26:59 +02:00
Dmitriy Ryajov
a5b051a14f
fix: use exception `msg` field in logs
2019-12-03 01:26:59 +02:00
Dmitriy Ryajov
9e437a681b
fix: revert to proper exception handling
2019-12-03 01:26:59 +02:00
cheatfate
dde8c01448
Workaround for https://github.com/nim-lang/Nim/issues/12764 .
...
Remove usage of getCurrentException().
2019-11-28 20:58:56 +02:00
Dmitriy Ryajov
fe3c1d45bb
misc: removing unneded accessors
2019-11-06 12:13:21 -06:00
Ștefan Talpalaru
f3fc763895
daemonapi: direct p2pd's output to parent's streams
2019-11-05 21:16:56 +01:00
cheatfate
76fee97724
Fix MultiAddress.init(PeerID) compilation problem.
2019-11-04 20:57:17 +02:00
cheatfate
833e8c9406
Fix overflow problem.
...
Fix compilation warnings.
2019-10-29 20:51:48 +02:00
Dmitriy Ryajov
400218ba01
fixing remaining overflows in BE/LE
2019-10-29 11:02:28 -06:00
Dmitriy Ryajov
541f0f2a41
avoid buffer overflow on BE/LE conversions
2019-10-29 10:10:19 -06:00
cheatfate
cc0d45c482
Add protoAddress() utility procedure.
2019-10-29 15:38:37 +02:00
Dmitriy Ryajov
21af081066
remove init in favor of upstream init
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
a4431a86a1
fixing windows build issue
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
0b77c8c6e3
add init from TransportAddress to multiaddress
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ae7b354819
unused directory
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9e16fd020e
use PeerID `==` for comparisons
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
018da65708
compare peer string ids, rather than keys
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
3194e3ac5b
fixes after building with 0.20.2
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
bb277d4b56
readLp should warn on ignored exceptions
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
464d75d159
minor style fixes
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
2066e81658
set default timeout to 10 secs
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
1d4b51413e
option to allow triggering own handlers on publish
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
37d7a03fba
use a timed cache in floodsub
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
e98ec9fd49
use doAssert for release asserts
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f190c155d3
don't throw on missing peer
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
3b9d34116d
decrease floodsub traffic
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f9a04dbb9d
todo: poll should allow for timeouts
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
cfd31c5251
properly handle empty messages
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
6cfbf2c124
don't send messages to self
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
a01d1db6a0
prperly close and chronos connections
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
968fd85d49
properly exit readLoop on error/stream end
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
34d1a641de
cleanup/test pubsub
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f8872dd51c
make log topics consistent
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
3c5b77affa
correctly identify incoming connection
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
68eb142a7c
end multiplex read loop on exception
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
cc2072d290
always cleanup connection on exit
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
c7606ce2eb
readMsg should never return an empty buffer
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f7e5c8ee30
bring back commented tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
bc46a76029
connection should raise on invalid varint
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
15bae7bec5
use `result` instead of buffer
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f5508be544
yield to chronos in mplex loop
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
b5c4eaf9b8
don't use result in oneliners
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
06b759e049
review cleanup
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
5ae37cac98
no more buffer resets on len prefixed reads
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
dd3f85965b
should not handle transport exceptions
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
99869aa8b8
don't reset buffer on exception
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
6baf089bb4
move tomultiaddr to wire.nim
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f3afe0a6ce
feat: allow ussage of 0 addrs
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9862064234
changed copyright year
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
5b3f93ba1c
feat: allow multiple handlers per topic in pubsub
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
2f31fc6940
misc: TODO for timeouts and message limit in mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
b270515bb3
feat: make private/public keys Option[T]
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
663ce6c589
misc: nimpretty
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
054085620c
logging: switch debug for trace in most cases
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
011df568b7
finished integrating secio
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
3eb0cdd5f7
misc
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9bb892de69
wip: integrating and testing secio
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
27e4516796
fix breaking test
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
b47dc89589
wip: integrating secio
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ea142f0e6d
proxy connection through secio
2019-10-11 08:15:24 +09:00
cheatfate
2d00f6a6db
wip: integrating secio
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f2fb92d1af
properly close transport
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
c98e8929bc
reverting change
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f39a5cf5e4
temporary fix, until pub/priv keys are Option[T]
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ef89e98479
renaming channel
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
88079a2864
incomming connection handling flow and tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
68d50a97f8
properly initialize hashsets
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9f3b80b60c
got pubsub working without signing
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
80267e81ec
exit main loop correctly
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
8920cd7d60
misc: pubsub/floodsub and logging
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
4d9444afe9
proper server startup sequence
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
5f2d944545
add muxer handler and properly set stream peerinfo
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
32b74c5d1c
add logging context
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
875ec346e6
adding bool suport to protobuf methods
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
003b72ec27
adding bool
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
67e10d0747
set mplex channel peerInfo correctly
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
4f158d4e83
adding pubsub interface to switch
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
41aad2e7e6
make peerId an Option[T]
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
177eb71ffa
wip: floodsub initial implementation
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
74d2aea63d
adding chronicles topics
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
827a8caba6
wip: modeling floodsub
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
bb648c87e1
wip: modeling secio
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
525646dcc1
handle secure managers correctly
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
fc36de21c1
handle exceptions correctly
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
2febd47e0d
don't complete futures twise
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
d27ea5d228
whip initial secio
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
435c69633f
adding chronicles logging
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
5bc8e7e7b1
remap exceptions
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
116051208a
make patterns runtime only
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
93a9fd203a
fix: multistream tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
e31966b6f8
fix: switch, with identify and mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
701e048ee6
add debug logging
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
54d740949e
debug logging
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
0c416e757e
make gcsafe
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ddd266f584
small helper to aid in debugging
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
b5dcfa9bc4
first attempt to proper channel cleanup
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
65ce1a93fc
prevent interleaved channel data
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
8c76799d9e
make writes sequential
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
e53c87e197
reworked to make msg reading sequential
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
c7fd08e002
use sane defaults
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
e5b782f094
mplex testing/interop
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
100f6220b3
dont clash with MultiCodec
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
6086038b01
raise correct LP exception on incomplete data
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ba843f4091
wip: interop testing
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
13ca6d253d
fix: multistream tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
962f58b436
avoid wrapping in several futures
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
13837619ed
plaintext security
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
8338a16aab
testing mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
b7f999d316
fixing several issues found while testing
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
dee46c4d37
remove unneded var
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
14d9150bbb
moving modules to apropriate directories
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
2f402d68e1
docs/wording
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
60245a065d
fix: don't call conn handler on dials
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
0f52a6e798
fix: switch and tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
6f8de062bb
fix: multistream
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
5237fc8b12
test multiple streams with mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
746987156d
reworked switch
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
0b784c5b58
don't pass stream handler through contructor
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
3cd19ddc47
change root object from ref to obj
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f8a942ed67
make peerinfo an optional type
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9889bd9cbf
return proto string from select, instead of bool
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
cc595f7947
allow selecting one of many protos in identify
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
a7e5fde6f7
check identify request against known pubkey
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ec351cc2b0
misc: cleanup mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
6ce2782e5c
add missing test
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
80c68206f7
documentation
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9b485b3082
feat: half closed channels
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
6058a3fc69
split mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
1b02c5e4e4
don't block main loop
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
36917c5adc
disable switch tests temporarily
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
a0aaa29739
wip: mplex and mplex tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
cc26d4e040
prevent already closed errors
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
8549ce76bb
add init proc to bufferedstream
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
917e0553e1
fix: tests and docs
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
96cd7bcf50
wip: mplex tests
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
b26d1ac23a
cosmetic
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
93bd7f623c
remove generic constructor
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
9bc7043f48
export read size const
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
cae4910427
add custom new proc
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
40e75a7635
remove useless methods
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
4edd2c9f8a
mplex move everything to one file
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ad1eaffdd6
make writeHandler async
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
d63e0c003b
add proper pragmas
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
5177bbb492
fix: return on read error
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
2c3c23c7c6
documentation
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
f3cc6fbef0
use correct names
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
7bd1e04d6f
wip: modeling mplex
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
7f0627378c
misc
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
fceea14aa5
adding bufferred stream
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
cbf0f4f186
move streams to own dir
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
ef7a0b41e4
add switch tests to native
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
0f3d024c0a
avoid unnecesary temps
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
18b9765525
fix: cleaning up based on jangko review
2019-10-11 08:15:24 +09:00
Dmitriy Ryajov
db1aad3df0
adding switch e2e test
2019-10-11 08:15:24 +09:00