* adding an upgraded event to conn
* set stopped flag asap
* trigger upgradded event on conn
* set concurrency limit for accepts
* backporting semaphore from tcp-limits2
* export unittests module
* make params explicit
* tone down debug logs
* adding semaphore tests
* use semaphore to throttle concurent upgrades
* add libp2p scope
* trigger upgraded event before any other events
* add event handler for connection upgrade
* cleanup upgraded event on conn close
* make upgrades slot release rebust
* dont forget to release slot on nil connection
* misc
* make sure semaphore is always released
* minor improvements and a nil check
* removing unneeded comment
* make upgradeMonitor a non-closure proc
* make sure the `upgraded` event is initialized
* handle exceptions in accepts when stopping
* don't leak exceptions when stopping accept loops
* streamline socket read/write hot path
This avoids some unnecessary memory copying on the hot path of noise /
mplex, as well as getting rid of a few futures - profiling shows that
this is one of the main culprits of small memory allocations, which
makes sense - this is where gossip fan-out happens.
* fewer futures (and corresponding closures) when sending lpchannel
messages
* avoid data copies when encrypting and framing noise messages
* avoid copying tuple when reading noise data (poor c codegen)
* fix setting eof flag in secure read
* write noise frames in one go
...and closing secure socket once is enough
* break stream tracking by type
* use closeWithEOF to await wrapped stream
* fix cancelation leaks
* fix channel leaks
* logging
* use close monitor and always call closeUnderlying
* don't use closeWithEOF
* removing close monitor
* logging
* move pubsub of off switch, pass switch into pubsub
* use join on lpstreams
* properly cleanup up failed peers
* fix tests
* fix peertable hasPeerId
* fix tests
* rework sending, remove helpers from pubsubpeer, unify in broadcast
* further split broadcast into send
* use send where appropriate
* use formatIt
* improve trace
Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
* reuse single RNG instance for all crypto key generation
* use foolproof rng
* initRng -> newRng (because it's ref)
* fix test
* imports/exports, chat fix
* fix rsa
* imports and exports
* work around threadvar issue
* fixup
* mac workaround test
* consolidate reading in lpstream
* remove debug echo
* throw if not enough bytes where read
* tune log level
* set eof flag
* test readExactly to fail on not enough bytes
* count published messages
* don't call `switch.dial` in `subscribeToPeer`
* add secureconn constructor
* close in the correct order
* concurent dial lock and track in/out conns better
* make tests pass
* add todo comment
* disconect peers that open too many connections
* wip
* do connection and muxer tracking in one place
* prevent nil pointer in observers
* drop connections when peers is over max
* prevent channel leaks
* don't use closure to handle channel
* Remove noise padding payload (spec removed it)
* add log scope in secure
* avoid defect array out of range in switch secure when "na"
* improve identify traces
* wip noise fixes
* noise protobuf adjustments (trying)
* add more debugging messages/traces, improve their actual contents
* re-enable ID check in noise
* bump go daemon tag version
* bump go daemon tag version
* enable noise in daemonapi
* interop testing, (both secio and noise will be tested)
* azure cache bump (p2pd)
* CI changes
- Travis: use Go 1.14
- azure-pipelines.yml: big cleanup
- Azure: bump cache keys
- build 64-bit p2pd on 32-bit Windows
- install both Mingw-w64 architectures
* noise logging fixes
* alternate testing between noise and secio
* increase timeout to avoid VM errors in CI (multistream tests)
* refactor heartbeat management in gossipsub
* remove locking within heartbeat
* refactor heartbeat management in gossipsub
* remove locking within heartbeat
Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
* count published messages
* don't call `switch.dial` in `subscribeToPeer`
* don't use delegation in connection
* move connection out to own file
* don't breakout on reset
* make sure to call close on secured conn
* add lpstream tracing
* don't breackdown by conn id
* fix import
* remove unused lable
* reset connection on exception
* add additional metrics for skipped messages
* check for nil in secure.close
* call write until all is written out
* add comments to lpchannel fields
* add an eof flag to signal which end closed
* wip: rework with proper half-closed
* add eof and closed handling
* propagate closes to piped
* call parent close
* moving bufferstream trackers out
* move writeLock to bufferstream
* move writeLock out
* remove unused call
* wip
* rebasing master
* fix mplex tests
* wip
* fix bufferstream after backport
* wip
* rename to differentiate from chronos tracker
* close connection on chronos close
* make reset request asyncCheck
* fix channel cleanup
* misc
* don't use read
* fix backports
* make noise work again
* proper exception handling
* don't reraise just yet
* add convenience templates
* dont double wrap
* use async pragma
* fixes after backporting
* muxer owns connection
* remove on transport close cleanup
* revert back allread
* adding some todos
* read from stream
* inc count before closing
* rebasing master
* rebase master
* use correct exception type
* use try/finally insted of defer
* fix compile in trace mode
* reset channels on mplex close
* Add chronos trackers and used them to sanitize resource disposal
* Chronos trackers for transport tests wip
* No more chronos leaks in testtransport
* Make tcp transport and test more robust when closing
* Test async leaking tracking wip
* Fix a regression in wire connect
* Add chronos trackers to more tests and sanitize resource closure
* Wip fixing floodsub tests
* Floodsub wip
* Made floodsub basically deterministic, hit a nim bug with captures tho
* Wrap up floodsub tests refactor
* Wrapping up
* Add allFuturesThrowing utility
* Fix missing allFuturesThrowing in noise tests!
* Make tests green
* attempt fixing gossipsub failing cases
* Make sure to check also fanout in waitSub
* More verbose traces
* Gossipsub test improvments
* Refactor TcpTransport remove asyncCheck
* Add Connection trackers
* Add stricter connection tracking, wip mplex fix
* More asynccheck removal, in order to avoid connection leaks
* bump chronicles requirement
* Enable tracker dump to check CI output
* Wait for more futures in testmplex
* Remove tracker dump messages
* add tryAndWarn utility, fix mplex issue with go interop
* All allFuturesThrowing to directchat too
* make sure to cleanup on transport close
* add stream metrics
- just BufferStream and Connection are tracked, for now
- flag checking is enforced more strictly in close(), since it became
clear that instances are closed multiple times
* add "metrics" dependency
and sort the list
* Make traces less verbose with shortHexDump utility
* Rename shortHexDump into shortLog
* Improve shortLog, add shortLog for crypto keys
* Add proper shortLog implementations in messages
* Start ChaCha20Poly1305 integration (BearSSL)
* Add Curve25519 (BearSSL) required operations for noise
* Fix curve mulgen iterate/derive
* Fix misleading header
* Add chachapoly proper test
* Curve25519 integration tests (failing, something is wrong)
* Add few converters, finish c25519 integration tests
* Remove implicit converters
* removed internal globals
* Start noise implementation
* Fix public() using proper bear mulgen
* Noise protocol WIP
* Noise progress
* Add a quick nim version of HKDF
* Converted hkdf to iterator, useful for noise
* Noise protocol implementation progress
* Noise progress
* XX handshake almost there
* noise progress
* Noise, testing handshake with test vectors
* Noise handshake progress, still wrong somewhere!
* Noise handshake success!
* Full verified noise XX handshake completed
* Fix and rewrite test to be similar to switch one
* Start with connection upgrade
* Switch chachapoly to CT implementations
* Improve HKDF implementation
* Use a type insted of tuple for HandshakeResult
* Remove unnecessary Let
* More cosmetic fixes
* Properly check randomBytes result
* Fix chachapoly signature
* Noise full circle (altho dispatcher is nil cursed)
* Allow nil aads in chachapoly routines
* Noise implementation up to running full test
* Use bearssl HKDF as well
* Directly use bearssl rng for curve25519 keys
* Add a (disabled/no CI) noise interop test server
* WIP on fixing interop issues
* More fixes in noise implementation for interop
* bump chronos requirement (nimble)
* Add a chachapoly test for very small size payloads
* Noise, more tracing
* Add 2 properly working noise tests
* Fix payload packing, following the spec properly (and not go version but
rather rust)
* Sanity, replace discard with asyncCheck
* Small fixes and optimization
* Use stew endian2 rather then system endian module
* Update nimble deps (chronos)
* Minor cosmetic/code sanity fixes
* Noise, handle Nonce max
* Noise tests, make sure to close secured conns
* More polish, improve code readability too
* More polish and testing again which test fails
* Further polishing
* Restore noise tests
* Remove useless Future[void]
* Remove useless CipherState initializer
* add a proper read wait future in second noise test
* Remove noise generic secure implementation for now
* Few fixes to run eth2 sim
* Add more debug info in noise traces
* Merge size + payload write in sendEncryptedMessage
* Revert secure interface, add outgoing property directly in newNoise
* remove sendEncrypted and receiveEncrypted
* Use openarray in chachapoly and curve25519 helpers
Dmitriy Ryajov
Jacek Sieka
Giovanni Petrantoni
Giovanni Petrantoni
Eugene Kabanov
Giovanni Petrantoni
Ștefan Talpalaru