Commit Graph

517 Commits

Author SHA1 Message Date
diegomrsantos af5299f26c
Create an ObservedAddrManager and add an AddressMapper in AutonatService and AutoRelayService (#871)
Co-authored-by: Tanguy <tanguy@status.im>
2023-03-24 15:42:49 +00:00
Tanguy 8d5ea43e2b
Upgrade flow refactoring (#807) 2023-03-08 12:30:19 +01:00
Tanguy c1a3bd8fee
Fix pubsub CI logs (#861) 2023-03-01 16:59:44 +01:00
diegomrsantos ddeb7b3bd4
Handle when peers ask each other at the same time (#865) 2023-02-21 17:49:41 +01:00
Ludovic Chenut 0012b639c8
Fix testrelay (#860) 2023-02-15 11:18:42 +01:00
Tanguy f14ada3dcf
Move tests flags from 'nimble test' to 'config.nims' (#852) 2023-02-10 11:32:21 +01:00
diegomrsantos 444b837923
Autonat doesn't ask an incoming peer (#857) 2023-02-09 17:40:04 +01:00
diegomrsantos f89bd0c77c
Autonat dials dns addrs (#856) 2023-02-09 16:53:46 +01:00
diegomrsantos e68186373b
Dialing addrs concurrently in autonat (#855) 2023-02-07 18:51:17 +01:00
diegomrsantos 266c7b117a
Add anyCompleted proc (#853) 2023-02-07 18:50:42 +01:00
Ludovic Chenut 0e28d3b828
Add the peerId to the transport.dial (#842) 2023-01-31 12:46:10 +01:00
diegomrsantos 4ace70d53b
Connect is able to force a new connection (#849) 2023-01-25 11:19:03 +01:00
Tanguy ca19f8fdbf
Autonat service: handle connections limits (#846)
Co-authored-by: diegomrsantos <diego@status.im>
2023-01-24 17:04:42 +01:00
Tanguy 351bda2b56
Add expected connections to connmngr (#845)
Co-authored-by: diegomrsantos <diego@status.im>
2023-01-23 22:28:39 +00:00
Tanguy c11772c94e
Happy new year! (#847) 2023-01-20 15:47:40 +01:00
Ludovic Chenut 489c115132
Autorelay service (#819) 2023-01-17 16:18:38 +01:00
diegomrsantos 9f658c151e
Autonat refactoring (#834) 2023-01-06 11:14:38 +01:00
diegomrsantos e304ad0f7e
Remove unnecessary async (#836) 2023-01-05 15:02:52 +01:00
diegomrsantos 5e3323d43f
More autonat tests (#833) 2022-12-23 19:10:15 +00:00
diegomrsantos 9532bff983
Ignore unknown answers (#831)
Co-authored-by: Tanguy <tanguy@status.im>
2022-12-23 16:49:25 +01:00
diegomrsantos 676786b00e
Add Autonat timeout (#829)
Co-authored-by: Ludovic Chenut <ludovic@status.im>
2022-12-22 19:29:31 +00:00
diegomrsantos d521c57b82
Handle dial error correctly (#830) 2022-12-22 17:33:59 +01:00
diegomrsantos 67ef25fae0
Autonat service (#814)
Co-authored-by: Tanguy <tanguy@status.im>
2022-12-16 11:32:00 +00:00
Tanguy 3863a4cd21
Allow connect with known PeerId in MA (#817) 2022-12-08 16:11:55 +00:00
Tanguy 64cbbe1e0a
Limit number of streams per protocol per peer (#811) 2022-12-01 12:20:40 +01:00
diegomrsantos b3d9360dfc
Starting switch two times does not crash (#810) 2022-11-29 15:21:51 +00:00
Tanguy 1711c204ea
Fix: big messages in pubsub (#808) 2022-11-24 16:43:45 +00:00
Alvaro Revuelta 192cac6254
Store missing pubkey in PeerStore [KeyBook] (#804) 2022-11-21 10:55:25 +01:00
Dmitriy Ryajov 8c2eca18dc
Allow passing max message size (#800)
Co-authored-by: Tanguy <tanguy@status.im>
2022-11-15 14:01:14 +01:00
diegomrsantos d9305bda84
Add Tor Transport support (#765) 2022-11-04 19:20:23 +01:00
Tanguy 4bce8f38c9
Remove sleepAsync from tests/testswitch (#792) 2022-10-27 23:10:24 +00:00
Tanguy dc13ff81d3
Revert "Temporarily remove failing test (#788)" & fix tests (#789)
Co-authored-by: Ludovic Chenut <ludovic@status.im>
2022-10-26 11:07:31 +02:00
Tanguy 2e12c7ab73
Temporarily remove failing test (#788) 2022-10-21 16:59:53 +02:00
Tanguy 7b103e02f2
Allow public address mapping (#767) 2022-10-20 12:22:28 +02:00
lchenut 32233d36c8
Discovery interface (#783)
Co-authored-by: Tanguy <tanguy@status.im>
2022-10-03 22:00:00 +00:00
lchenut 1c99aca054
RendezVous Protocol (#751) 2022-09-30 10:41:04 +02:00
Tanguy fa5d102370
Better dnsaddr resolving (#753) 2022-09-26 11:03:24 +02:00
diegomrsantos a56c3bc296
Make observedAddr optional (#772)
Co-authored-by: Tanguy <tanguy@status.im>
2022-09-22 21:55:59 +02:00
Tanguy 5e7e009445
Move relay & autonat to connectivity folder (#769) 2022-09-15 09:43:40 +02:00
Tanguy 72abe822c0
Fix switch failed start (#770) 2022-09-15 07:06:32 +00:00
lchenut 4d8b50d24c
Specify EOF error (#759) 2022-09-14 10:58:41 +02:00
diegomrsantos d8a9e93ff7
Add onion3 multiaddr support (#764) 2022-09-08 17:10:11 +02:00
Tanguy abbeaab684
Keep connection alive when peer doesn't support pubsub (#754) 2022-09-06 13:20:42 +02:00
Jacek Sieka dfbfbe6eb6
allow connection to a peer with unknown PeerId (#756)
Co-authored-by: Tanguy <tanguy@status.im>
2022-09-05 14:31:14 +02:00
Tanguy 3ffc03ed16
Gossipsub: Rebalance mesh immediately when peer sub/unsub (#719) 2022-09-02 10:24:54 +02:00
Miran 14d2c3f51e
Configuration refactoring (#732)
move -d:nimRawSetjmp to nim.cfg
switch .cfg to .nims
add --skipParentCfg

Co-authored-by: Tanguy <tanguy@status.im>
2022-09-01 14:00:22 +02:00
lchenut 2332813873
Autonat protocol (#739) 2022-08-23 17:49:07 +02:00
Tanguy 124a7a5ffe
Add missing stub logger (#752) 2022-08-03 19:04:17 +02:00
Tanguy 2d864633ea
ConnManager connection tracking refacto (#749) 2022-08-03 16:48:19 +02:00
Miran 2fbe82bf9d
make it more Nim 1.4+ compatible (#742) 2022-08-03 13:33:19 +02:00
lchenut 34c2fb8787
Circuit relay v2 (#717) 2022-08-01 14:31:22 +02:00
tersec 1e598a0239
topicIDs -> topicIds; PubsubPeerEvent -> PubSubPeerEvent; messageIDs -> messageIds (#748) 2022-07-27 17:14:05 +00:00
tersec 4ca1c2d7ed
--styleCheck:error (#743)
* --styleCheck:error

* PeerID -> PeerId

* other libp2p styleCheck changes

* MessageID -> messageId; topicID -> topicId
2022-07-27 11:47:50 +00:00
lchenut 912873f8b3
Replace table by seq for storing muxers (#741)
Replace table by seq for stocking muxers
2022-07-22 12:54:09 +02:00
Tanguy 533e39ef94
Yamux (#704)
Co-authored-by: Ludovic Chenut <ludovic@status.im>
2022-07-04 15:19:21 +02:00
Tanguy 150fafbee8
Faster CI (#724)
* Only compile once in CI
* Fix codecov fetch
2022-07-01 20:20:42 +02:00
Tanguy d0523fdc9d
Documentation (#716) 2022-07-01 20:19:57 +02:00
lchenut 0ece5eaf12
Start/stop protocols (#730)
Starting/stopping a switch now starts/stops all protocols mounted on that switch
2022-06-30 11:21:33 +02:00
Tanguy a7e335e1bb
Remove Br prefix from BearSSL (#728)
ref status-im/nim-bearssl#27
2022-06-16 10:08:52 +02:00
Tanguy 60becadcf9
Peer store refacto (#700)
There is now a global PeerStore structure (instead of having one for libp2p, one for waku, etc)

The user can create custom books for new types easily

Also add a pruning system to remove dead peers
2022-05-25 12:12:57 +02:00
Tanguy d4ff1c88e9
Less warnings (#710) 2022-05-24 15:10:57 +02:00
lchenut 13503f3799
Circuit relay v1 (#670)
Circuit relay v1
2022-05-18 10:19:37 +02:00
Tanguy 991549f391
Gossipsub scoring fixes (#709)
* Use decayInterval as a scoring heartbeat period
* Take mesh delivery window into account
2022-05-11 10:38:43 +02:00
lchenut 32ca1898d9
Gossipsub: Put Peer Exchange behind a flag (#715)
Add a flag to enable Peer Exchange in Gossipsub (disabled by default)
2022-05-10 10:39:43 +02:00
Tanguy 9ba5c069c8
Update SPR when the PeerInfo changes (#711) 2022-04-13 09:35:28 +02:00
Tanguy c97befb387
Add tests for gossipsub direct peers (#707) 2022-04-12 14:03:31 +00:00
tersec fc6b8f46f1
encrypt and decrypt empty sequences (#713)
* encrypt and decrypt empty sequences

* use assign in curve25519
2022-04-12 10:41:48 +00:00
Tanguy c7504d2446
Gossipsub peer exchange (#647)
* Signed envelopes and routing records
* Send signed peer record as part of identify (#649)
* Add SPR from identify to new peer book (#657)
* Send & receive gossipsub PX
* Add Signed Payload

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-03-14 09:39:30 +01:00
Eric Mastro 44a7260f07
fixes from #688 (#697)
* tests: invert message logic on expect from #688
* fix: export pubsub_errors for backward compatibility
2022-02-24 17:32:20 +01:00
Tanguy c09d032133
Allow force dial (#696) 2022-02-24 17:31:47 +01:00
Tanguy f98bf612bd
Fix tests of #638 2022-02-21 18:14:43 +01:00
Tanguy bc318084f4
GS: Publish to fanout when mesh unhealthy (#638)
* Send to fanout when mesh unhealthy

* don't use fanout when floodPublish
2022-02-21 16:22:08 +01:00
Eric Mastro 3b718baa97
feat: allow msgIdProvider to fail (#688)
* feat: allow msgIdProvider to fail

Closes: #642.

Changes the return type of the msgIdProvider to `Result[MessageID, string]` so that message id generation can fail.

String error type was chosen as this `msgIdProvider` mainly because the failed message id generation drops the message and logs the error provided. Because `msgIdProvider` can be externally provided by library consumers, an enum didn’t make sense and a object seemed to be overkill. Exceptions could have been used as well, however, in this case, Result ergonomics were warranted and prevented wrapping quite a large block of code in try/except.

The `defaultMsgIdProvider` function previously allowed message id generation to fail silently for use in the tests: when seqno or source peerid were not valid, the message id generated was based on a hash of the message data and topic ids. The silent failing was moved to the `defaultMsgIdProvider` used only in the tests so that it could not fail silently in applications.

Unit tests were added for the `defaultMsgIdProvider`.

* Change MsgIdProvider error type to ValidationResult
2022-02-21 16:04:17 +01:00
Tanguy 1a97d0a2f5
Validate pubsub subscriptions (#627)
* Check topic before subscribing
* Block subscribe to invalid topics
2022-01-14 12:40:30 -06:00
Ștefan Talpalaru e72d03bc78
CI: test with multiple Nim versions by default (#684)
* CI: test with 1.2 & devel by default
* Skip buggy windows websocket test
2022-01-10 12:29:52 +01:00
Tanguy f3dee6865c
Chronos strict exception tracking (#652)
* Enable chronos strict exception tracking ( -d:chronosStrictException )
2022-01-05 16:27:33 +01:00
Dmitriy Ryajov dffe4bed45
mitigate high traffic - drop faulty peers (#460)
Only allow 1 connection per peer

Co-authored-by: Tanguy <tanguy@status.im>
2022-01-04 11:21:24 +00:00
Tanguy fb0d10b6fd
Gossipsub: process messages concurrently (#680)
* Gossip sub: process messages concurrently

* Retries for flaky test
2021-12-27 11:17:00 +01:00
Tanguy df566e69db
Fixes for style check (#676) 2021-12-16 11:05:20 +01:00
Tanguy 47a35e26d7
Typo: s/unsubcribeBackoff/unsubscribeBackoff (#675) 2021-12-14 10:50:57 +01:00
Etan Kissling 2373ee0061
harden `ecnist` byte export against uninitialized key (#671)
Currently, `ecnist`'s `toBytes` and `getBytes` methods operate only on
properly initialized keys. If an un-initialized key is given, an
`IndexError` may be raised if the key's `xlen` / `qlen` property is
larger than the maximum buffer size. This patch hardens those functions
to report a proper error in that case.
Note that the library functions called by `init` and `initRaw` already
reject data that does not have the expected length, so these new checks
should not be reachable in practice.
2021-12-13 18:46:25 +00:00
Tanguy 8fe44e35c2
Fix flaky tests (#669)
* Fix push identify flaky test
* Fix mplex flaky tests
* Fix switch flaky test
2021-12-13 11:25:33 +01:00
Eric Mastro fffa7e8cc2
fix: remove returned Futures from switch.start (#662)
* fix: remove returned Futures from switch.start

The proc `start` returned a seq of futures that was mean to be awaited by the caller. However, the start proc itself awaited each Future before returning it, so the ceremony requiring the caller to await the Future, and returning the Futures themselves was just used to handle errors. But we'll give a better way to handle errors in a future revision

Remove `switch.start` return type (implicit `Future[void]`)

Update tutorials and examples to reflect the change.

* Raise error during failed transport

Replaces logging of error, and adds comment that it should be replaced with a callback in a future PR.
2021-12-03 19:23:12 +01:00
Tanguy 6893bd9dbb
Customizable gossipsub backoff on unsubscribe (#665)
* Customizable gossipsub backoff on unsubscribe
* change default to 5s
2021-12-02 14:47:40 +00:00
Tanguy 0dfac6fce7
Signed envelopes and routing records (#656) 2021-11-24 14:03:40 -06:00
Dmitriy Ryajov 73168b6eae
Add support for multiple addresses to transports (#598)
* add test for multiple local addresses

* allow transports to listen on multiple addrs

* fix tcp transport accept

* check switch addrs are correct

* switch test to port 0

* close accepted peers on close

* ignore CancelledError in transport accept

* test ci

* only accept in accept loop

* avoid accept greedyness

* close acceptedPeers

* accept doesn't crash on cancelled fut

* add common transport test

* close conn on handling failure

* close accepted peers in two steps

* test for macos

* revert accept greedyness

* fix dialing cancel

* test chronos fix

* add ws

* ws cancellation

* small fix

* remove chronos blocked test

* fix testping

* Fix transport's switch start (like #609)

* bump chronos

* Websocket: handle both ws & wss

Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
Co-authored-by: Tanguy <tanguy@status.im>
2021-11-24 14:01:12 -06:00
Tanguy 6f779c47c8
Gossipsub: don't send to peers seen during validation (#648)
* Gossipsub: don't send to peers seen during validation

* Less error prone code

* add metric

* Fix metric

* remove dangling code test

* address comments

* don't allocate memory
2021-11-14 09:08:05 +01:00
Tanguy c92125a1a4
Integrate dns resolving (#615)
* integrate dns

* give hostname to transport dial

* add hostname test

* switched to websock master

* Add dnsaddr dial test w multiple transports
2021-11-08 13:02:03 +01:00
Tanguy e1d96a0f4d
Remove isWire (#640) 2021-10-28 19:11:31 +02:00
Tanguy 5885e03680
Add maxMessageSize option to pubsub (#634)
* Add maxMessageSize option to pubsub
* Switched default to 1mb
2021-10-25 12:58:38 +02:00
Tanguy 846baf3853
Various cleanups part 1 (#632)
* raise -> raise exc
* replace stdlib random with bearssl
* object init -> new
* Remove deprecated procs
* getMandatoryField
2021-10-25 10:26:32 +02:00
Tanguy 3669b90ceb
Fix WS observed address (#631)
* Fix WS observed address

* Unify tcptransport & wstransport
2021-10-14 13:16:34 +02:00
Menduist d02735dc46
Remove peer info (#610)
Peer Info is now for local peer data only.
For other peers info, use the peer store.

Previous reference to peer info are replaced with the peerid
2021-09-08 11:07:46 +02:00
Tanguy Cizain 8cddfde837
Rename getKey -> getPublicKey (#621)
* rename getKey to getPublicKey

* use publicKey directly in gossipsub

* update error messages
2021-09-02 12:03:40 +02:00
Tanguy Cizain 62ca6dd9a0
Add missing test to testnative (#618)
* add missing tests

* fix testping
2021-09-01 08:38:24 +02:00
Tanguy Cizain f274bfe19d
DNS Addresses handling (#580)
* add 'dns' multiaddr protocol

* multiaddr: isWire is true for DNS protocols

* resolve dns on connect

* fix typo

* add dns test

* update resolveDns error handling

* handle multiple dns entries

* start of new resolver

* working dns resolver

* use the DnsResolver

* fix json logs

* small overhaul

* fix dns implem in lp2p

* update dnsclient repo

* add dns test to testnative

* dummy dns server for ut

* better mocked

* moved resolving to transport

* moved mockresolver to libp2p

* test resolve in switch test

* try multiple txt & track leaks

* raise e

* catchable error instead of exception

* save failed dns server

* moved resolve back to dialer

* remove nameresolver from dialer
2021-08-18 09:40:12 +02:00
Tanguy Cizain af3be7966b
Websocket Transport (#593)
* start of websocket transport

* more ws tests

* switch to common test

* add close to wsstream

* update ws & chronicles version

* cleanup

* removed multicodec

* clean ws outgoing connections

* renamed to websock

* removed stream from logs

* renamed ws to websock

* add connection closing test to common transport

* close incoming connection on ws stop

* renamed testwebsocket.nim -> testwstransport.nim

* removed raise todo

* split out/in connections

* add wss to tests

* Fix tls (#608)

* change log level

* fixed issue related to stopping

some cosmetic cleanup

* use `allFutures` to stop/close things

Prevent potential race conditions when stopping two or more transports

* misc

* point websock to server-case-object branch

* interop test with go

* removed websock version specification

* add daemon -> native ws test

* fix & test closed read/write

* update readOnce, thanks jangko

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2021-08-03 15:48:03 +02:00
markspanbroek 8b438142ce
Fixes failing peer store test in Nim 1.4.8 (#612)
* Fixes failing test in Nim 1.4.8

This tests failed because the order in which the elements
of a hashset are added to a seq is non-deterministic.

Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-28 12:06:18 +02:00
Tanguy Cizain c1b2d45d1b
Track ChronosStream in test (#605)
* track chronos stream in tests

* use chronosstreamtrackername
2021-07-13 13:53:08 +02:00
Tanguy Cizain 26e47d7da5
Various transports improvement (#594)
* little transport cleanup

* rename TcpTransport.init -> TcpTransport.new

* moved transport e2e to common file

* remove localAddress

* rename testtransport -> testtcptransport

* add checktrackers to commontransports

* removed multicodec from transports
2021-06-30 10:59:30 +02:00