01339c991f
Don't use and expose directly secp types ( #183 )
...
* Don't use and expose directly secp types
* Reuse same secp type names
2020-05-19 14:48:55 +02:00
c219100e64
Use results and no exceptions in chacha and curve25519 ( #182 )
2020-05-19 10:22:49 +02:00
f8029e7359
use sha256 digest as cache keys ( #135 )
...
* use sha256 digest as cache keys
* rebasing master
2020-05-18 14:49:49 -06:00
9cf1fd0216
remove generic constructor and expose serverflags ( #176 )
...
* remove generic constructor and expose serverflags
* fix transport constructor
* fix merge issues
2020-05-18 13:04:05 -06:00
773b738c12
don't track Connection, track StreamTransport ( #177 )
...
* don't track Connection, track StreamTransport
* make tests more deterministic
2020-05-18 11:05:34 -06:00
5583168965
Track all connections ( #175 )
...
* adding to do for future refactor
* no mix conn and transport stopping or it will race
* don't use intermediary vars
2020-05-18 09:08:10 -06:00
1819502fb5
Cleanup - tests and logging ( #178 )
...
* make async for proper exception handling
* tryAndWarn msg messes up Exception msg
* misc: comment out tracker dumps
* cleanup mplex tests
* more informative errors
* give CI time to run
* revert change, bacause it causes races
2020-05-18 07:49:49 -06:00
7dcb807f64
Crypto utilities resultification ( #150 )
2020-05-18 07:25:55 +02:00
167f42ed45
Remove read ( #171 )
...
* use readExactly
* remove `read`
* remove read
* no more `read`
2020-05-14 22:02:05 -06:00
69abf5097d
handle a few exceptions ( #170 )
...
* handle a few exceptions
Some of these are maybe too aggressive, but in return, they'll log
their exception - more refactoring needed to sort this out - right now
we get crashes on unhandled exceptions of unknown origin
* during connection setup
* while closing channels
* while processing pubsubs
* catch exceptions that are raised and don't try to catch exceptions that are not raised
* propagate cancellederror
* one more
* more
* more
* make interop tests less fragile
* Raise expiration time in gossipsub fanout test for slow CI
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-05-14 21:56:56 -06:00
005e088405
Properly track and close mplex handlers ( #166 )
...
* Properly track and close mplex handlers
* Avoid verbose warnings
* Fix tryAndWarn trace issue
* Handle LPEOF in lpchannel close
2020-05-12 14:45:32 +02:00
618e01eba3
don't crash on double peerinfo close ( #167 )
2020-05-11 21:05:24 +02:00
3053f03814
fix varint issues
...
* fixes #111
2020-05-11 09:12:23 -06:00
87e1f3c61f
fix trace logging
2020-05-09 13:56:07 +02:00
fbf640794b
move channel size constant
2020-05-09 08:48:26 +02:00
6196d56fc2
check for nil observers
2020-05-08 15:44:18 -06:00
ccd019b328
use stream directly in chronosstream ( #163 )
...
* use stream directly in chronosstream
for now, chronos.AsyncStream is not used to provide any features on top
of chronos.Stream, so in order to simplify the code, chronosstream can
be used directly.
In particular, the exception handling is broken in the current
chronosstream - opening and closing the stream is simplified this way as
well.
A future implementation that actually takes advantage of the AsyncStream
features would wrap AsyncStream instead as a separate lpstream
implementation, leaving this one as-is.
* work around chronos exception type issue
2020-05-08 22:10:06 +02:00
c889224012
Add PubSub observer+ hooks (they can modify as well)
2020-05-08 13:31:52 -06:00
268253ea18
remove Chronos type from public API
2020-05-08 13:23:36 -06:00
313f9b0952
use a Transport.serverFlags attribute
2020-05-08 13:23:36 -06:00
c480e65055
use set[ServerFlags] params instead of hardcoded flags
2020-05-08 13:23:36 -06:00
125843af7d
TcpTransport.listen(): enable SO_REUSEADDR
...
in order to avoid failures when restarting the process while the OS is
keeping open sockets from previous runs
2020-05-08 13:23:36 -06:00
1efada474c
remove readLoop in secure protocols ( #162 )
...
* remove readLoop in secure protocols, fix security issues
* fix Defect on remote sending 0-byte noise/secio message
* remove msglen from `write` (unused)
* simplify SecureConn data flow
* document some control-flow issues
* unify exception behaviour across noise and secio
* secio would not raise on mac/decryption errors
* fix compile error
2020-05-07 14:37:46 -06:00
330da51819
removals ( #159 )
...
* remove unused stream methods
* reimplement some of them with proc's
* remove broken tests
* Error->Defect for defect
* warning fixes
2020-05-06 18:31:47 +02:00
6da4d2af48
Pubsub signatures flags ( #161 )
...
* add verify signature flag
* add sign flag to enable/disable msg signing
* moving internal tests out to their own file
* cleanup nimble file
* remove unneeded tests
* move pubsub tests out
* fix tests
2020-05-06 11:26:08 +02:00
e1928456a7
avoid newlines in $
...
they mess with debug prints and logging (same reason why $(seq) doesn't
print them
2020-04-28 10:59:53 -06:00
290ba712e9
Fix MultiAddress.protoAddress() bug for fixed arrays.
...
Add tests for it.
2020-04-28 14:43:44 +03:00
8a22c073c7
Fix secure/noise securing explicitly, added noise to pubsub tests
2020-04-24 14:33:08 -06:00
917b5f5c84
Add MultiAddress.init(integer) for tcp,udp,dccp,sctp protocols.
...
Add tests for it.
2020-04-23 08:10:17 -06:00
1c4d72f5e3
Use Result construct in minasn1 ( #144 )
2020-04-23 14:10:20 +02:00
6ae92eb21a
Hotfix noise read usage, replace with readExactly
2020-04-22 23:36:59 +09:00
d96372f820
Minor tweaking in errors.nim in order to give priority to Defect
2020-04-21 22:10:47 +09:00
4c6a123d31
Add chronos trackers and used them to sanitize resource disposal ( #131 )
...
* Add chronos trackers and used them to sanitize resource disposal
* Chronos trackers for transport tests wip
* No more chronos leaks in testtransport
* Make tcp transport and test more robust when closing
* Test async leaking tracking wip
* Fix a regression in wire connect
* Add chronos trackers to more tests and sanitize resource closure
* Wip fixing floodsub tests
* Floodsub wip
* Made floodsub basically deterministic, hit a nim bug with captures tho
* Wrap up floodsub tests refactor
* Wrapping up
* Add allFuturesThrowing utility
* Fix missing allFuturesThrowing in noise tests!
* Make tests green
* attempt fixing gossipsub failing cases
* Make sure to check also fanout in waitSub
* More verbose traces
* Gossipsub test improvments
* Refactor TcpTransport remove asyncCheck
* Add Connection trackers
* Add stricter connection tracking, wip mplex fix
* More asynccheck removal, in order to avoid connection leaks
* bump chronicles requirement
* Enable tracker dump to check CI output
* Wait for more futures in testmplex
* Remove tracker dump messages
* add tryAndWarn utility, fix mplex issue with go interop
* All allFuturesThrowing to directchat too
* make sure to cleanup on transport close
2020-04-21 10:24:42 +09:00
e8b33c64fa
secp: use upstream secp convenience api ( #141 )
...
* secp: use upstream secp convenience api
2020-04-17 12:51:13 +02:00
eaa73ae6e8
add stream metrics ( #136 )
...
* add stream metrics
- just BufferStream and Connection are tracked, for now
- flag checking is enforced more strictly in close(), since it became
clear that instances are closed multiple times
* add "metrics" dependency
and sort the list
2020-04-14 15:27:07 +02:00
7723403b1f
debug prints ( #132 )
...
* debug prints
* CI: enable stack traces
* Azure: better NimBinaries cache key
* CI changes
- Azure: remove Linux target
- Travis: add ARM64 target
* uglify the code in order to save 12 bytes per LPStream object
2020-04-14 15:21:16 +02:00
2b823bde68
secp: update ( #138 )
2020-04-12 19:03:08 +02:00
303ec297da
Start removing allFutures ( #125 )
...
* Start removing allFutures
* More allfutures removal
* Complete allFutures removal except legacy and tests
* Introduce table values copies to prevent error
* Switch to allFinished
* Resolve TODOs in flood/gossip
* muxer handler, log and re-raise
* Add a common and flexible way to check multiple futures
2020-04-11 13:08:25 +09:00
f4740c8b8e
fix trace runs in connection
2020-04-07 14:55:05 -06:00
00fbc9246e
fix nil condition
2020-04-07 12:16:59 -06:00
6cbcc7859e
reduse usssage of asyncCheck
2020-04-07 12:16:59 -06:00
bd49a35e0a
formatting
2020-04-07 12:16:59 -06:00
976164ba3c
proper connection cleanup
2020-04-07 12:16:59 -06:00
0c8dde15e7
remove unused field
2020-04-07 12:16:59 -06:00
4ee1b4a66d
connection closing tests
2020-04-07 12:16:59 -06:00
5a657c9264
disable storing connections on internal table
2020-04-07 12:16:59 -06:00
87b35de83e
close connections
2020-04-07 12:16:59 -06:00
bae4713a85
small cleanup
2020-04-07 12:16:59 -06:00
8dda6c289d
more connection closes to fix leaks
2020-04-07 12:16:59 -06:00
e4303110a6
properly close connections
2020-04-07 12:16:59 -06:00
35a48fa560
Re-enable gossipsub internal tests when running CI
...
minor bonus: add a link in the comments about bearssl issue with
callbacks
2020-04-07 22:07:00 +09:00
a576069305
Chachapoly new bearssl fixes
2020-04-07 17:52:19 +09:00
f7af2c2e5c
Fix possible crash in tcptransport when server is nil
2020-04-06 12:56:17 +09:00
3514733060
Fix table assertion, edited while iterating
...
(the fix is not so nice.. adds plenty of allocations, but for now should
be ok)
2020-04-05 01:19:10 +09:00
e39bf0a4cf
Mplex: Add the ability to send any size payload ( #123 )
...
* Mplex: Add the ability to send any size payload
* Ensure size of coder header
2020-04-03 17:26:46 +02:00
7f8090b166
SecioConn.readMessage(): fix the exception types
2020-04-02 19:07:14 +02:00
5285f0d091
Fix/misc ( #116 )
...
* only check for payload size
* only subscribe if connection succeeded
* fix failing test
* check that the strem is active before openning
* msg type should not be > than 0x7
* fix tests
* check max against enum val
2020-03-29 08:28:48 -06:00
a35ecc6227
reverting connection changes
2020-03-28 09:29:43 -06:00
3effb95f10
close underlying bufferstream in lpchannel
2020-03-28 09:29:43 -06:00
56392913f9
Allow to specify a read size in readLp ( #117 )
...
* Allow to specify a read size in readLp
Also fix secio fragments + test
* Revert readLp changes, use just `read` in test
2020-03-26 15:06:47 +09:00
33ceefb209
Remove nim random usage from noise
2020-03-26 15:04:46 +09:00
b132eae939
Add secio oversized payload fragmentation
...
Cannot be used because Connection Read size hard limit tho
2020-03-25 16:53:35 +09:00
1550bea1cc
Support arbitrary stream write sizes in noise
2020-03-25 16:10:11 +09:00
4199508f17
readMplexVarint should return uint64
2020-03-24 15:41:40 -06:00
2de98751ae
fix: use exact types for mplex id
2020-03-24 15:41:40 -06:00
1f5d994700
Fix compilation errors introduced by latest chronos.
2020-03-24 09:48:05 +02:00
3d10513b3e
Properly await the handshake end in secure handler
2020-03-24 15:34:02 +09:00
0a3e4a764b
Less verbose traces ( #112 )
...
* Make traces less verbose with shortHexDump utility
* Rename shortHexDump into shortLog
* Improve shortLog, add shortLog for crypto keys
* Add proper shortLog implementations in messages
2020-03-23 15:03:36 +09:00
b1a34f478e
Avoid 2 async consecutive writes in noise HS
2020-03-18 20:40:51 +09:00
c02fca25f8
Noise ( #90 )
...
* Start ChaCha20Poly1305 integration (BearSSL)
* Add Curve25519 (BearSSL) required operations for noise
* Fix curve mulgen iterate/derive
* Fix misleading header
* Add chachapoly proper test
* Curve25519 integration tests (failing, something is wrong)
* Add few converters, finish c25519 integration tests
* Remove implicit converters
* removed internal globals
* Start noise implementation
* Fix public() using proper bear mulgen
* Noise protocol WIP
* Noise progress
* Add a quick nim version of HKDF
* Converted hkdf to iterator, useful for noise
* Noise protocol implementation progress
* Noise progress
* XX handshake almost there
* noise progress
* Noise, testing handshake with test vectors
* Noise handshake progress, still wrong somewhere!
* Noise handshake success!
* Full verified noise XX handshake completed
* Fix and rewrite test to be similar to switch one
* Start with connection upgrade
* Switch chachapoly to CT implementations
* Improve HKDF implementation
* Use a type insted of tuple for HandshakeResult
* Remove unnecessary Let
* More cosmetic fixes
* Properly check randomBytes result
* Fix chachapoly signature
* Noise full circle (altho dispatcher is nil cursed)
* Allow nil aads in chachapoly routines
* Noise implementation up to running full test
* Use bearssl HKDF as well
* Directly use bearssl rng for curve25519 keys
* Add a (disabled/no CI) noise interop test server
* WIP on fixing interop issues
* More fixes in noise implementation for interop
* bump chronos requirement (nimble)
* Add a chachapoly test for very small size payloads
* Noise, more tracing
* Add 2 properly working noise tests
* Fix payload packing, following the spec properly (and not go version but
rather rust)
* Sanity, replace discard with asyncCheck
* Small fixes and optimization
* Use stew endian2 rather then system endian module
* Update nimble deps (chronos)
* Minor cosmetic/code sanity fixes
* Noise, handle Nonce max
* Noise tests, make sure to close secured conns
* More polish, improve code readability too
* More polish and testing again which test fails
* Further polishing
* Restore noise tests
* Remove useless Future[void]
* Remove useless CipherState initializer
* add a proper read wait future in second noise test
* Remove noise generic secure implementation for now
* Few fixes to run eth2 sim
* Add more debug info in noise traces
* Merge size + payload write in sendEncryptedMessage
* Revert secure interface, add outgoing property directly in newNoise
* remove sendEncrypted and receiveEncrypted
* Use openarray in chachapoly and curve25519 helpers
2020-03-17 13:30:01 +01:00
761b221b75
don't trace the payload
2020-03-12 08:01:43 -06:00
5c234f704d
added 1 byte jitter test
2020-03-12 08:01:43 -06:00
406b79887d
add jitter tolerance to mplex
2020-03-12 08:01:43 -06:00
82af623641
use correct shift
2020-03-11 09:58:24 -06:00
07695b2260
use `readExactly` not `read` to read all bytes
2020-03-11 09:58:24 -06:00
52cb7ced04
prefer endians2 ( #98 )
2020-03-10 21:40:52 +01:00
df29ac760e
remove missing include
2020-03-09 13:26:38 -06:00
fbe54e5c5a
move SecureConn to secure.nim
2020-03-09 13:26:38 -06:00
19095a0f85
move `secure` to base as well
2020-03-09 13:26:38 -06:00
d1c6591b8a
Fix and refactoring of some procedures which are able to return nil as result ( #97 )
...
* Fix do not return nil as result.
* Fix mplex test to properly raise.
2020-03-09 13:26:38 -06:00
c3e791db0d
remove unused imports
2020-03-09 13:26:38 -06:00
f1a297f0f3
add initiator flag to handshake
2020-03-09 13:26:38 -06:00
d1d133319e
change handleConn to be a proc
2020-03-09 13:26:38 -06:00
9aa9e97602
feat: move connection handling logic to base class
2020-03-09 13:26:38 -06:00
871d8d7478
remove white space
2020-03-09 13:26:38 -06:00
5701d937c8
Signed variable integers fixes. ( #96 )
...
* Fix signed varints.
Add tests for signed varints.
Remove some casts to allow usage at compile time.
* Fix vsizeof() on 32bit platforms.
* Add `hint` and `zint` types for proper signed integer encoding.
* Fix varint related bugs.
* Update requirements.
* Fix interop tests because of fixed readLine.
* Add putVarint, getVarint and tests.
2020-03-06 20:19:43 +01:00
381630f185
Fix and refactoring of some procedures which are able to return nil as result ( #97 )
...
* Fix do not return nil as result.
* Fix mplex test to properly raise.
2020-03-04 21:45:14 +02:00
ffc3b04222
read should await for data on an empty buff
2020-03-01 04:06:42 -06:00
91d2f52f4e
Add some utility procedures to MultiAddress. ( #94 )
...
* Add some utility procedures to MultiAddress.
* Do not use result and add more operations to be more effective for compiler.
2020-02-27 19:26:42 +02:00
85d4baca95
await connection
2020-02-25 17:52:08 -06:00
946b9580c0
don't use nil result
2020-02-25 17:52:08 -06:00
52c270dcf9
warn on failed pubsub subscribe
2020-02-25 17:52:08 -06:00
eb49d4b218
no empty proto dials and add connect method
2020-02-25 17:52:08 -06:00
9efc08cb2f
fix : #83
2020-02-24 08:58:02 -06:00
6a7f9f058c
address review comments
2020-02-21 09:21:06 -06:00
fbcef69891
implicitelly dial pubsub if enabled
2020-02-21 09:21:06 -06:00
cb156f3260
Fix public() using proper bear mulgen
2020-02-18 09:35:14 -06:00
93a480e6f3
removed internal globals
2020-02-18 09:35:14 -06:00
e0e371994c
Remove implicit converters
2020-02-18 09:35:14 -06:00
f039853a19
Add few converters, finish c25519 integration tests
2020-02-18 09:35:14 -06:00
99a169b65d
Curve25519 integration tests (failing, something is wrong)
2020-02-18 09:35:14 -06:00
2103aa59f3
Add chachapoly proper test
2020-02-18 09:35:14 -06:00
6ba274de27
Fix misleading header
2020-02-18 09:35:14 -06:00
f47fb628f0
Fix curve mulgen iterate/derive
2020-02-18 09:35:14 -06:00
8dec368878
Add Curve25519 (BearSSL) required operations for noise
2020-02-18 09:35:14 -06:00
71998ffc67
Start ChaCha20Poly1305 integration (BearSSL)
2020-02-18 09:35:14 -06:00
1a987a9c5b
fix: delete mplex channel
2020-02-16 13:51:03 -06:00
65ed2c7cee
make code more robust
2020-02-16 11:31:35 -06:00
9023bf786d
remove sleeps
2020-02-16 11:31:35 -06:00
acdaeb8f5d
working out synchronization issues
2020-02-16 11:31:35 -06:00
b1dd564c32
make sure we don't exit the handler prematurely
2020-02-16 11:31:35 -06:00
f9ad113d11
don't send close message if remote closed
2020-02-16 11:31:35 -06:00
f767614827
remove unnecesary async lock
2020-02-16 11:31:35 -06:00
934c858542
increase timeout to allow floodsub to finish
2020-02-16 11:31:35 -06:00
26a4510377
remove unnecesary nil check in peerinfo
2020-02-16 11:31:35 -06:00
0349165830
typo
2020-02-16 11:31:35 -06:00
7f8eb0272e
cleanup and fix tests
2020-02-16 11:31:35 -06:00
6316b07e81
don't crash when stringifying nil peerinfo
2020-02-16 11:31:35 -06:00
4faf35c52c
throw exception if handshake failed
2020-02-16 11:31:35 -06:00
f4e9bc8bfb
remove sleepAsync for synchronization
2020-02-16 11:31:35 -06:00
53e163abf2
check for size bounds when reading varints
2020-02-16 11:31:35 -06:00
94fc4e6fd2
don't use sleeps for synchronization
2020-02-16 11:31:35 -06:00
39dc9ad8a3
Revert "Implement timeout in mplex in order to read reliably close state"
...
This reverts commit 314cb6e6bf
2020-02-12 11:57:38 -05:00
11bd55308f
Revert "Properly use already specified MaxReadWriteTime as timeout"
...
This reverts commit 8f5dd75e7f
2020-02-12 11:57:38 -05:00
ea0d167285
Fix max size write mplex test
2020-02-12 11:57:38 -05:00
cf5dd27e57
Properly use already specified MaxReadWriteTime as timeout
2020-02-12 11:57:38 -05:00
4034dfae19
Implement timeout in mplex in order to read reliably close state
2020-02-12 11:57:38 -05:00
136ac3f550
Use already existing error for channel push over size
2020-02-12 11:57:38 -05:00
f9cbdc252f
Improve connection.nim read error clarity when buffer overflow
2020-02-12 11:57:38 -05:00
120ba0d528
Add a safety check on message size limit when pushing new data in mplex
2020-02-12 11:57:38 -05:00
3531ebf772
Fix typo
2020-02-12 11:57:38 -05:00
5b6f4bf5fa
Export MultiAddress pattern procedures.
2020-02-12 12:04:12 +02:00
540e79a430
Add peer lifetime feature for PeerInfo. ( #77 )
...
* Add peer lifetime feature for PeerInfo.
Refactor peerinfo to use openarrays instead of sequences.
Fix tests and examples to use arrays instead of sequences.
* Add access to lifetime Future[T] itself.
2020-02-11 12:53:39 -05:00
23712ecf3b
Lazy channels ( #78 )
...
* Implemented lazy stream opening for mplex connections
* Properly fix newStream usage
* Make lazy channel open optional
* Add Lazy channel test
* Cleanup mplex test
* Move lazyness properly into LPChannel
* Connection writeLp back to proc
2020-02-11 12:30:36 -05:00
88a030d8fb
fix: removing timeouts from conn
2020-02-05 20:38:43 +01:00
2232ca598e
don't timeout in pubsub
2020-02-04 17:59:57 +01:00
1bd933cd5a
More precise tracing
2020-02-04 17:27:32 +01:00
7bd305471c
Make sure the library can compile with json logging in trace mode
2020-02-04 15:17:39 +01:00
691efaa8a1
dont use result
2020-01-30 09:30:08 -06:00
171d60fb52
fix: reraise exception in readLp
2020-01-30 09:30:08 -06:00
06886290dd
Add links to specifications to varint comments.
2020-01-29 11:14:25 +02:00
d902127595
Fix LibP2P varint implementation to follow latest specification. ( #65 )
...
* Fix LibP2P varint implementation to decode only minimal valus.
2020-01-28 16:29:42 +01:00
007b0219f3
use ECDSA as defaults, faster in tests
2020-01-09 12:55:21 -06:00
667691f784
send messages in batches
2020-01-09 12:55:21 -06:00
cd8961cfb9
close and cleanup connections
2020-01-09 12:55:21 -06:00
c6561b8851
add timeouts to connection/secio
2020-01-09 12:55:21 -06:00
107e71203d
Fix #61 .
2020-01-09 17:38:10 +02:00
58cdefac21
don't expose private api
2019-12-24 10:35:35 -06:00
0fb1f1c5b8
strenghten pubsub interop testing
2019-12-24 10:35:35 -06:00
8714c66353
allow empty protocol string in dial
2019-12-24 10:35:35 -06:00
68cc57669e
Feat/pubsub validators ( #58 )
...
* feat: adding validator hooks to pubsub
* expose add/remove validators on switch
* do less unnecessary copyng
2019-12-16 23:24:03 -06:00
b6b0cdea98
fix: inconsistent error handling ( #56 )
...
* fix: inconsistent error handling
* fixing a few more returns
* remov reduntant returns
* feat: dont sleep in readLoop
2019-12-13 09:25:07 -06:00
e0aae6d8ac
Moved base32, 58 and 64 to stew
2019-12-13 16:00:39 +02:00
293a219dbe
Cleanup ( #55 )
...
* fix: don't allow replacing pubkey
* fix: several small improvements
* removing pubkey setter
* improove error handling
* remove the use of Option[T] if not needed
* don't use optional
* fix-ci: temporarily pin p2pd to a working tag
* fix example to comply with latest changes
* bumping p2pd again to a higher version
2019-12-10 14:50:35 -06:00
77e866d29a
Address review comments
2019-12-09 19:17:29 +02:00
454f658ba8
Fixes and tweaks related to the beacon node integration
...
* Bugfix: Dialing an already connected peer may lead to crash
* Introduced a standard_setup module allowing to instantiate
the `Switch` object in an easier manner.
* Added `Switch.disconnect(peer)`
* Trailing space removed (sorry about polluting the diff)
2019-12-08 23:58:43 +02:00
5f6fcc3d90
extract public and private keys fields from peerid ( #44 )
...
* extract public and private keys fields from peerid
* allow assigning a public key
* cleaned up TODOs
* make pubsub prefix a const
* public key should be an `Option`
2019-12-07 10:36:39 -06:00
e623e70e7b
PubSub (Gossip & Flood) Implementation ( #36 )
...
This adds gossipsub and floodsub, as well as basic interop testing with the go libp2p daemon.
* add close event
* wip: gossipsub
* splitting rpc message
* making message handling more consistent
* initial gossipsub implementation
* feat: nim 1.0 cleanup
* wip: gossipsub protobuf
* adding encoding/decoding of gossipsub messages
* add disconnect handler
* add proper gossipsub msg handling
* misc: cleanup for nim 1.0
* splitting floodsub and gossipsub tests
* feat: add mesh rebalansing
* test pubsub
* add mesh rebalansing tests
* testing mesh maintenance
* finishing mcache implementatin
* wip: commenting out broken tests
* wip: don't run heartbeat for now
* switchout debug for trace logging
* testing gossip peer selection algorithm
* test stream piping
* more work around message amplification
* get the peerid from message
* use timed cache as backing store
* allow setting timeout in constructor
* several changes to improve performance
* more through testing of msg amplification
* prevent gc issues
* allow piping to self and prevent deadlocks
* improove floodsub
* allow running hook on cache eviction
* prevent race conditions
* prevent race conditions and improove tests
* use hashes as cache keys
* removing useless file
* don't create a new seq
* re-enable pubsub tests
* fix imports
* reduce number of runs to speed up tests
* break out control message processing
* normalize sleeps between steps
* implement proper transport filtering
* initial interop testing
* clean up floodsub publish logic
* allow dialing without a protocol
* adding multiple reads/writes
* use protobuf varint in mplex
* don't loose conn's peerInfo
* initial interop pubsub tests
* don't duplicate connections/peers
* bring back interop tests
* wip: interop
* re-enable interop and daemon tests
* add multiple read write tests from handlers
* don't cleanup channel prematurely
* use correct channel to send/receive msgs
* adjust tests with latest changes
* include interop tests
* remove temp logging output
* fix ci
* use correct public key serialization
* additional tests for pubsub interop
2019-12-05 20:16:18 -06:00
903e79ede1
Feat/conn cleanup ( #41 )
...
Backporting proper connection cleanup from #36 to align with latest chronos changes.
* add close event
* use proper varint encoding
* add proper channel cleanup in mplex
* add connection cleanup in secio
* tidy up
* add dollar operator
* fix tests
* don't close connections prematurely
* handle closing streams properly
* misc
* implement address filtering logic
* adding pipe tests
* don't use gcsafe if not needed
* misc
* proper connection cleanup and stream muxing
* re-enable pubsub tests
2019-12-03 22:44:54 -06:00
1df16bdbce
set log level to trace - not enabled by default
2019-12-02 18:43:21 -06:00
f9eed172d4
Don't handle Exception
2019-12-03 01:26:59 +02:00
26366f8b56
use `Exception` to emulate `getCurrentException()`
2019-12-03 01:26:59 +02:00
a5b051a14f
fix: use exception `msg` field in logs
2019-12-03 01:26:59 +02:00
9e437a681b
fix: revert to proper exception handling
2019-12-03 01:26:59 +02:00
dde8c01448
Workaround for https://github.com/nim-lang/Nim/issues/12764 .
...
Remove usage of getCurrentException().
2019-11-28 20:58:56 +02:00
fe3c1d45bb
misc: removing unneded accessors
2019-11-06 12:13:21 -06:00
f3fc763895
daemonapi: direct p2pd's output to parent's streams
2019-11-05 21:16:56 +01:00
76fee97724
Fix MultiAddress.init(PeerID) compilation problem.
2019-11-04 20:57:17 +02:00
833e8c9406
Fix overflow problem.
...
Fix compilation warnings.
2019-10-29 20:51:48 +02:00
400218ba01
fixing remaining overflows in BE/LE
2019-10-29 11:02:28 -06:00
541f0f2a41
avoid buffer overflow on BE/LE conversions
2019-10-29 10:10:19 -06:00
cc0d45c482
Add protoAddress() utility procedure.
2019-10-29 15:38:37 +02:00
21af081066
remove init in favor of upstream init
2019-10-11 08:15:24 +09:00
a4431a86a1
fixing windows build issue
2019-10-11 08:15:24 +09:00
0b77c8c6e3
add init from TransportAddress to multiaddress
2019-10-11 08:15:24 +09:00
ae7b354819
unused directory
2019-10-11 08:15:24 +09:00
9e16fd020e
use PeerID `==` for comparisons
2019-10-11 08:15:24 +09:00
018da65708
compare peer string ids, rather than keys
2019-10-11 08:15:24 +09:00
3194e3ac5b
fixes after building with 0.20.2
2019-10-11 08:15:24 +09:00
bb277d4b56
readLp should warn on ignored exceptions
2019-10-11 08:15:24 +09:00
464d75d159
minor style fixes
2019-10-11 08:15:24 +09:00
2066e81658
set default timeout to 10 secs
2019-10-11 08:15:24 +09:00
1d4b51413e
option to allow triggering own handlers on publish
2019-10-11 08:15:24 +09:00
37d7a03fba
use a timed cache in floodsub
2019-10-11 08:15:24 +09:00
e98ec9fd49
use doAssert for release asserts
2019-10-11 08:15:24 +09:00
f190c155d3
don't throw on missing peer
2019-10-11 08:15:24 +09:00
3b9d34116d
decrease floodsub traffic
2019-10-11 08:15:24 +09:00
f9a04dbb9d
todo: poll should allow for timeouts
2019-10-11 08:15:24 +09:00
cfd31c5251
properly handle empty messages
2019-10-11 08:15:24 +09:00
6cfbf2c124
don't send messages to self
2019-10-11 08:15:24 +09:00
a01d1db6a0
prperly close and chronos connections
2019-10-11 08:15:24 +09:00
968fd85d49
properly exit readLoop on error/stream end
2019-10-11 08:15:24 +09:00
34d1a641de
cleanup/test pubsub
2019-10-11 08:15:24 +09:00
f8872dd51c
make log topics consistent
2019-10-11 08:15:24 +09:00
3c5b77affa
correctly identify incoming connection
2019-10-11 08:15:24 +09:00
68eb142a7c
end multiplex read loop on exception
2019-10-11 08:15:24 +09:00
cc2072d290
always cleanup connection on exit
2019-10-11 08:15:24 +09:00
c7606ce2eb
readMsg should never return an empty buffer
2019-10-11 08:15:24 +09:00
f7e5c8ee30
bring back commented tests
2019-10-11 08:15:24 +09:00
bc46a76029
connection should raise on invalid varint
2019-10-11 08:15:24 +09:00
15bae7bec5
use `result` instead of buffer
2019-10-11 08:15:24 +09:00
f5508be544
yield to chronos in mplex loop
2019-10-11 08:15:24 +09:00
b5c4eaf9b8
don't use result in oneliners
2019-10-11 08:15:24 +09:00
06b759e049
review cleanup
2019-10-11 08:15:24 +09:00
5ae37cac98
no more buffer resets on len prefixed reads
2019-10-11 08:15:24 +09:00
dd3f85965b
should not handle transport exceptions
2019-10-11 08:15:24 +09:00
99869aa8b8
don't reset buffer on exception
2019-10-11 08:15:24 +09:00
6baf089bb4
move tomultiaddr to wire.nim
2019-10-11 08:15:24 +09:00
f3afe0a6ce
feat: allow ussage of 0 addrs
2019-10-11 08:15:24 +09:00
9862064234
changed copyright year
2019-10-11 08:15:24 +09:00
5b3f93ba1c
feat: allow multiple handlers per topic in pubsub
2019-10-11 08:15:24 +09:00
2f31fc6940
misc: TODO for timeouts and message limit in mplex
2019-10-11 08:15:24 +09:00
b270515bb3
feat: make private/public keys Option[T]
2019-10-11 08:15:24 +09:00
663ce6c589
misc: nimpretty
2019-10-11 08:15:24 +09:00
054085620c
logging: switch debug for trace in most cases
2019-10-11 08:15:24 +09:00
011df568b7
finished integrating secio
2019-10-11 08:15:24 +09:00
3eb0cdd5f7
misc
2019-10-11 08:15:24 +09:00
9bb892de69
wip: integrating and testing secio
2019-10-11 08:15:24 +09:00
27e4516796
fix breaking test
2019-10-11 08:15:24 +09:00
b47dc89589
wip: integrating secio
2019-10-11 08:15:24 +09:00
ea142f0e6d
proxy connection through secio
2019-10-11 08:15:24 +09:00
2d00f6a6db
wip: integrating secio
2019-10-11 08:15:24 +09:00
f2fb92d1af
properly close transport
2019-10-11 08:15:24 +09:00
c98e8929bc
reverting change
2019-10-11 08:15:24 +09:00
f39a5cf5e4
temporary fix, until pub/priv keys are Option[T]
2019-10-11 08:15:24 +09:00
ef89e98479
renaming channel
2019-10-11 08:15:24 +09:00
88079a2864
incomming connection handling flow and tests
2019-10-11 08:15:24 +09:00
68d50a97f8
properly initialize hashsets
2019-10-11 08:15:24 +09:00
9f3b80b60c
got pubsub working without signing
2019-10-11 08:15:24 +09:00
80267e81ec
exit main loop correctly
2019-10-11 08:15:24 +09:00
8920cd7d60
misc: pubsub/floodsub and logging
2019-10-11 08:15:24 +09:00
4d9444afe9
proper server startup sequence
2019-10-11 08:15:24 +09:00
5f2d944545
add muxer handler and properly set stream peerinfo
2019-10-11 08:15:24 +09:00
32b74c5d1c
add logging context
2019-10-11 08:15:24 +09:00
875ec346e6
adding bool suport to protobuf methods
2019-10-11 08:15:24 +09:00
003b72ec27
adding bool
2019-10-11 08:15:24 +09:00
67e10d0747
set mplex channel peerInfo correctly
2019-10-11 08:15:24 +09:00
4f158d4e83
adding pubsub interface to switch
2019-10-11 08:15:24 +09:00
41aad2e7e6
make peerId an Option[T]
2019-10-11 08:15:24 +09:00
177eb71ffa
wip: floodsub initial implementation
2019-10-11 08:15:24 +09:00
74d2aea63d
adding chronicles topics
2019-10-11 08:15:24 +09:00
827a8caba6
wip: modeling floodsub
2019-10-11 08:15:24 +09:00
bb648c87e1
wip: modeling secio
2019-10-11 08:15:24 +09:00
525646dcc1
handle secure managers correctly
2019-10-11 08:15:24 +09:00
fc36de21c1
handle exceptions correctly
2019-10-11 08:15:24 +09:00
2febd47e0d
don't complete futures twise
2019-10-11 08:15:24 +09:00
d27ea5d228
whip initial secio
2019-10-11 08:15:24 +09:00
435c69633f
adding chronicles logging
2019-10-11 08:15:24 +09:00
5bc8e7e7b1
remap exceptions
2019-10-11 08:15:24 +09:00
116051208a
make patterns runtime only
2019-10-11 08:15:24 +09:00
93a9fd203a
fix: multistream tests
2019-10-11 08:15:24 +09:00
e31966b6f8
fix: switch, with identify and mplex
2019-10-11 08:15:24 +09:00
701e048ee6
add debug logging
2019-10-11 08:15:24 +09:00
54d740949e
debug logging
2019-10-11 08:15:24 +09:00
0c416e757e
make gcsafe
2019-10-11 08:15:24 +09:00
ddd266f584
small helper to aid in debugging
2019-10-11 08:15:24 +09:00
b5dcfa9bc4
first attempt to proper channel cleanup
2019-10-11 08:15:24 +09:00
65ce1a93fc
prevent interleaved channel data
2019-10-11 08:15:24 +09:00
8c76799d9e
make writes sequential
2019-10-11 08:15:24 +09:00
e53c87e197
reworked to make msg reading sequential
2019-10-11 08:15:24 +09:00
c7fd08e002
use sane defaults
2019-10-11 08:15:24 +09:00
e5b782f094
mplex testing/interop
2019-10-11 08:15:24 +09:00
100f6220b3
dont clash with MultiCodec
2019-10-11 08:15:24 +09:00
6086038b01
raise correct LP exception on incomplete data
2019-10-11 08:15:24 +09:00
ba843f4091
wip: interop testing
2019-10-11 08:15:24 +09:00
13ca6d253d
fix: multistream tests
2019-10-11 08:15:24 +09:00
962f58b436
avoid wrapping in several futures
2019-10-11 08:15:24 +09:00
13837619ed
plaintext security
2019-10-11 08:15:24 +09:00
8338a16aab
testing mplex
2019-10-11 08:15:24 +09:00
b7f999d316
fixing several issues found while testing
2019-10-11 08:15:24 +09:00
dee46c4d37
remove unneded var
2019-10-11 08:15:24 +09:00
14d9150bbb
moving modules to apropriate directories
2019-10-11 08:15:24 +09:00
2f402d68e1
docs/wording
2019-10-11 08:15:24 +09:00
60245a065d
fix: don't call conn handler on dials
2019-10-11 08:15:24 +09:00
0f52a6e798
fix: switch and tests
2019-10-11 08:15:24 +09:00
6f8de062bb
fix: multistream
2019-10-11 08:15:24 +09:00
5237fc8b12
test multiple streams with mplex
2019-10-11 08:15:24 +09:00
746987156d
reworked switch
2019-10-11 08:15:24 +09:00
0b784c5b58
don't pass stream handler through contructor
2019-10-11 08:15:24 +09:00
3cd19ddc47
change root object from ref to obj
2019-10-11 08:15:24 +09:00
f8a942ed67
make peerinfo an optional type
2019-10-11 08:15:24 +09:00
9889bd9cbf
return proto string from select, instead of bool
2019-10-11 08:15:24 +09:00
cc595f7947
allow selecting one of many protos in identify
2019-10-11 08:15:24 +09:00
a7e5fde6f7
check identify request against known pubkey
2019-10-11 08:15:24 +09:00
ec351cc2b0
misc: cleanup mplex
2019-10-11 08:15:24 +09:00
6ce2782e5c
add missing test
2019-10-11 08:15:24 +09:00
80c68206f7
documentation
2019-10-11 08:15:24 +09:00
9b485b3082
feat: half closed channels
2019-10-11 08:15:24 +09:00
6058a3fc69
split mplex
2019-10-11 08:15:24 +09:00
1b02c5e4e4
don't block main loop
2019-10-11 08:15:24 +09:00
36917c5adc
disable switch tests temporarily
2019-10-11 08:15:24 +09:00
a0aaa29739
wip: mplex and mplex tests
2019-10-11 08:15:24 +09:00
cc26d4e040
prevent already closed errors
2019-10-11 08:15:24 +09:00
8549ce76bb
add init proc to bufferedstream
2019-10-11 08:15:24 +09:00
917e0553e1
fix: tests and docs
2019-10-11 08:15:24 +09:00
96cd7bcf50
wip: mplex tests
2019-10-11 08:15:24 +09:00
b26d1ac23a
cosmetic
2019-10-11 08:15:24 +09:00
93bd7f623c
remove generic constructor
2019-10-11 08:15:24 +09:00
9bc7043f48
export read size const
2019-10-11 08:15:24 +09:00
cae4910427
add custom new proc
2019-10-11 08:15:24 +09:00
40e75a7635
remove useless methods
2019-10-11 08:15:24 +09:00
4edd2c9f8a
mplex move everything to one file
2019-10-11 08:15:24 +09:00
ad1eaffdd6
make writeHandler async
2019-10-11 08:15:24 +09:00
d63e0c003b
add proper pragmas
2019-10-11 08:15:24 +09:00
5177bbb492
fix: return on read error
2019-10-11 08:15:24 +09:00
2c3c23c7c6
documentation
2019-10-11 08:15:24 +09:00
f3cc6fbef0
use correct names
2019-10-11 08:15:24 +09:00
7bd1e04d6f
wip: modeling mplex
2019-10-11 08:15:24 +09:00
7f0627378c
misc
2019-10-11 08:15:24 +09:00
fceea14aa5
adding bufferred stream
2019-10-11 08:15:24 +09:00
cbf0f4f186
move streams to own dir
2019-10-11 08:15:24 +09:00
ef7a0b41e4
add switch tests to native
2019-10-11 08:15:24 +09:00
0f3d024c0a
avoid unnecesary temps
2019-10-11 08:15:24 +09:00
18b9765525
fix: cleaning up based on jangko review
2019-10-11 08:15:24 +09:00
db1aad3df0
adding switch e2e test
2019-10-11 08:15:24 +09:00
79129ea6d1
`handles` should be called on concrete transport
2019-10-11 08:15:24 +09:00
3df92e0d67
wip: switch
2019-10-11 08:15:24 +09:00
152c1e3c14
close connections asyncronously
2019-10-11 08:15:24 +09:00
022a248ca3
reworked protocol
2019-10-11 08:15:24 +09:00
3d74a4c82a
add base upgrade method
2019-10-11 08:15:24 +09:00
71282fa442
make matcher func gcsafe
2019-10-11 08:15:24 +09:00
1cef1b803b
don't await twise
2019-10-11 08:15:24 +09:00
e2b04fc30d
add gcsafe pragma
2019-10-11 08:15:24 +09:00
ca4ccbe37f
don't use method as handler in protocol base
2019-10-11 08:15:24 +09:00
e7c3412d69
expose private and public keys
2019-10-11 08:15:24 +09:00
e5be1fcaee
wip: switch
2019-10-11 08:15:24 +09:00
5eced0abee
rename multistream
2019-10-11 08:15:24 +09:00
aa2dfababd
wip: identify
2019-10-11 08:15:24 +09:00
682fecc395
make things public
2019-10-11 08:15:24 +09:00
054bd365af
protocol handler method should receive context
2019-10-11 08:15:24 +09:00
b85d616a13
call the handler with the protocol context
2019-10-11 08:15:24 +09:00
a9c9788356
we don't need the switch in the protos after all
2019-10-11 08:15:24 +09:00
a7709b8afc
use gcsafe in closures
2019-10-11 08:15:24 +09:00
b56b005ab7
make object instead of ref
2019-10-11 08:15:24 +09:00
2d2c3ba977
identify message protobuf encode/decode
2019-10-11 08:15:24 +09:00
e77ebc2e5a
pretty
2019-10-11 08:15:24 +09:00
954a78fda0
remove useless error pragma
2019-10-11 08:15:24 +09:00
261204853e
rename readwrite to stream
2019-10-11 08:15:24 +09:00
f56d925f6f
rename supports to handles
2019-10-11 08:15:24 +09:00
e486b5e40c
make fields privae
2019-10-11 08:15:24 +09:00
90d9dcbce9
wip: identify and switch
2019-10-11 08:15:24 +09:00
ec86afebe0
modeling the switch
2019-10-11 08:15:24 +09:00
3d7f657ce8
rename readerwriter to stream
2019-10-11 08:15:24 +09:00
d23398f498
nimpretty
2019-10-11 08:15:24 +09:00
187645013f
fix: ls and other minor issues
2019-10-11 08:15:24 +09:00
139b8a752b
correctly close stream
2019-10-11 08:15:24 +09:00
47b6b89fc5
fix: call on reader
2019-10-11 08:15:24 +09:00
77aa11f0f3
removing bins
2019-10-11 08:15:24 +09:00
8a52cfa9b0
use consistent dispatcher object name
2019-10-11 08:15:24 +09:00
64e92eb9da
adding `na` test/functionality
2019-10-11 08:15:24 +09:00
Giovanni Petrantoni
Dmitriy Ryajov
Jacek Sieka
Giovanni Petrantoni
Ștefan Talpalaru
cheatfate
Zahary Karadjov
Yuriy Glukhov
cheatfate