Currently, `ecnist`'s `toBytes` and `getBytes` methods operate only on
properly initialized keys. If an un-initialized key is given, an
`IndexError` may be raised if the key's `xlen` / `qlen` property is
larger than the maximum buffer size. This patch hardens those functions
to report a proper error in that case.
Note that the library functions called by `init` and `initRaw` already
reject data that does not have the expected length, so these new checks
should not be reachable in practice.
* fix: remove returned Futures from switch.start
The proc `start` returned a seq of futures that was mean to be awaited by the caller. However, the start proc itself awaited each Future before returning it, so the ceremony requiring the caller to await the Future, and returning the Futures themselves was just used to handle errors. But we'll give a better way to handle errors in a future revision
Remove `switch.start` return type (implicit `Future[void]`)
Update tutorials and examples to reflect the change.
* Raise error during failed transport
Replaces logging of error, and adds comment that it should be replaced with a callback in a future PR.
* add test for multiple local addresses
* allow transports to listen on multiple addrs
* fix tcp transport accept
* check switch addrs are correct
* switch test to port 0
* close accepted peers on close
* ignore CancelledError in transport accept
* test ci
* only accept in accept loop
* avoid accept greedyness
* close acceptedPeers
* accept doesn't crash on cancelled fut
* add common transport test
* close conn on handling failure
* close accepted peers in two steps
* test for macos
* revert accept greedyness
* fix dialing cancel
* test chronos fix
* add ws
* ws cancellation
* small fix
* remove chronos blocked test
* fix testping
* Fix transport's switch start (like #609)
* bump chronos
* Websocket: handle both ws & wss
Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
Co-authored-by: Tanguy <tanguy@status.im>
* add 'dns' multiaddr protocol
* multiaddr: isWire is true for DNS protocols
* resolve dns on connect
* fix typo
* add dns test
* update resolveDns error handling
* handle multiple dns entries
* start of new resolver
* working dns resolver
* use the DnsResolver
* fix json logs
* small overhaul
* fix dns implem in lp2p
* update dnsclient repo
* add dns test to testnative
* dummy dns server for ut
* better mocked
* moved resolving to transport
* moved mockresolver to libp2p
* test resolve in switch test
* try multiple txt & track leaks
* raise e
* catchable error instead of exception
* save failed dns server
* moved resolve back to dialer
* remove nameresolver from dialer
* start of websocket transport
* more ws tests
* switch to common test
* add close to wsstream
* update ws & chronicles version
* cleanup
* removed multicodec
* clean ws outgoing connections
* renamed to websock
* removed stream from logs
* renamed ws to websock
* add connection closing test to common transport
* close incoming connection on ws stop
* renamed testwebsocket.nim -> testwstransport.nim
* removed raise todo
* split out/in connections
* add wss to tests
* Fix tls (#608)
* change log level
* fixed issue related to stopping
some cosmetic cleanup
* use `allFutures` to stop/close things
Prevent potential race conditions when stopping two or more transports
* misc
* point websock to server-case-object branch
* interop test with go
* removed websock version specification
* add daemon -> native ws test
* fix & test closed read/write
* update readOnce, thanks jangko
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
* little transport cleanup
* rename TcpTransport.init -> TcpTransport.new
* moved transport e2e to common file
* remove localAddress
* rename testtransport -> testtcptransport
* add checktrackers to commontransports
* removed multicodec from transports
* Connect & Peer event handlers now receive a peerinfo
* small peerstore refacto
* implement peerstore in switch
* changed PeerStore to final ref object
* revert libp2p/builders.nim
* Revisit Floodsub (#543)
Fixes#525
add coverage to unsubscribeAll and testing
* add mounted protos to identify message (#546)
* add stable/unstable auto bumps
* fix auto-bump CI
* merge nbc auto bump with CI in order to bump only on CI success
* put conditional locks on nbc bump (#549)
* Fix minor exception issues (#550)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
* fix nimbus ref for auto-bump stable's PR
* use a builder pattern to build the switch (#551)
* use a builder pattern to build the switch
* with with
* more refs
* builders (#559)
* More builders (#560)
* address some issues pointed out in review
* re-add to prevent breaking other projects
* mem usage cleanups for pubsub (#564)
In `async` functions, a closure environment is created for variables
that cross an await boundary - this closure environment is kept in
memory for the lifetime of the associated future - this means that
although _some_ variables are no longer used, they still take up memory
for a long time.
In Nimbus, message validation is processed in batches meaning the future
of an incoming gossip message stays around for quite a while - this
leads to memory consumption peaks of 100-200 mb when there are many
attestations in the pipeline.
To avoid excessive memory usage, it's generally better to move non-async
code into proc's such that the variables therein can be released earlier
- this includes the many hidden variables introduced by macro and
template expansion (ie chronicles that does expensive exception
handling)
* move seen table salt to floodsub, use there as well
* shorten seen table salt to size of hash
* avoid unnecessary memory allocations and copies in a few places
* factor out message scoring
* avoid reencoding outgoing message for every peer
* keep checking validators until reject (in case there's both reject and
ignore)
* `readOnce` avoids `readExactly` overhead for single-byte read
* genericAssign -> assign2
* More gossip coverage (#553)
* add floodPublish test
* test delivery via control Iwant/have mechanics
* fix issues in control, and add testing
* fix possible backoff issue with pruned routine overriding it
* fix control messages (#566)
* remove unused control graft check in handleControl
* avoid sending empty Iwant messages
* Split dialer (#542)
* extracting dialing logic to dialer
* exposing upgrade methods on transport
* cleanup
* fixing tests to use new interfaces
* add comments
* add base exception class and fix hierarchy
* fix imports
* Merge master (#555)
* Revisit Floodsub (#543)
Fixes#525
add coverage to unsubscribeAll and testing
* add mounted protos to identify message (#546)
* add stable/unstable auto bumps
* fix auto-bump CI
* merge nbc auto bump with CI in order to bump only on CI success
* put conditional locks on nbc bump (#549)
* Fix minor exception issues (#550)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
* fix nimbus ref for auto-bump stable's PR
* Split dialer (#542)
* extracting dialing logic to dialer
* exposing upgrade methods on transport
* cleanup
* fixing tests to use new interfaces
* add comments
* add base exception class and fix hierarchy
* fix imports
* `doAssert` is `ValueError` not `AssertionError`?
* revert back to `AssertionError`
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
* Builders (#558)
* use a builder pattern to build the switch (#551)
* use a builder pattern to build the switch
* with with
* more refs
* Merge master (#555)
* Revisit Floodsub (#543)
Fixes#525
add coverage to unsubscribeAll and testing
* add mounted protos to identify message (#546)
* add stable/unstable auto bumps
* fix auto-bump CI
* merge nbc auto bump with CI in order to bump only on CI success
* put conditional locks on nbc bump (#549)
* Fix minor exception issues (#550)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
* fix nimbus ref for auto-bump stable's PR
* Split dialer (#542)
* extracting dialing logic to dialer
* exposing upgrade methods on transport
* cleanup
* fixing tests to use new interfaces
* add comments
* add base exception class and fix hierarchy
* fix imports
* `doAssert` is `ValueError` not `AssertionError`?
* revert back to `AssertionError`
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
* `doAssert` is `ValueError` not `AssertionError`?
* revert back to `AssertionError`
* fix builders
* more builder stuff
* more builders
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
* builders (#559)
* More builders (#560)
* address some issues pointed out in review
* re-add to prevent breaking other projects
* Merge master (#555)
* Revisit Floodsub (#543)
Fixes#525
add coverage to unsubscribeAll and testing
* add mounted protos to identify message (#546)
* add stable/unstable auto bumps
* fix auto-bump CI
* merge nbc auto bump with CI in order to bump only on CI success
* put conditional locks on nbc bump (#549)
* Fix minor exception issues (#550)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
* fix nimbus ref for auto-bump stable's PR
* Split dialer (#542)
* extracting dialing logic to dialer
* exposing upgrade methods on transport
* cleanup
* fixing tests to use new interfaces
* add comments
* add base exception class and fix hierarchy
* fix imports
* `doAssert` is `ValueError` not `AssertionError`?
* revert back to `AssertionError`
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
* use a builder pattern to build the switch (#551)
* use a builder pattern to build the switch
* with with
* more refs
* Merge master (#555)
* Revisit Floodsub (#543)
Fixes#525
add coverage to unsubscribeAll and testing
* add mounted protos to identify message (#546)
* add stable/unstable auto bumps
* fix auto-bump CI
* merge nbc auto bump with CI in order to bump only on CI success
* put conditional locks on nbc bump (#549)
* Fix minor exception issues (#550)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
* fix nimbus ref for auto-bump stable's PR
* Split dialer (#542)
* extracting dialing logic to dialer
* exposing upgrade methods on transport
* cleanup
* fixing tests to use new interfaces
* add comments
* add base exception class and fix hierarchy
* fix imports
* `doAssert` is `ValueError` not `AssertionError`?
* revert back to `AssertionError`
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
* `doAssert` is `ValueError` not `AssertionError`?
* revert back to `AssertionError`
* fix builders
* more builder stuff
* more builders
Co-authored-by: Giovanni Petrantoni <7008900+sinkingsugar@users.noreply.github.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
Dmitriy Ryajov
Tanguy
Jacek Sieka
Etan Kissling
Eric Mastro
Tanguy
Zahary Karadjov
Miran