36 Commits

Author SHA1 Message Date
Danish Arora
477c2a5918
feat!: protocols filter peers as per configured shard (#1756)
* merge: master

* fix: tests

* update: interfafces

* rm: comments

* metadata: store peerIdStr instead of peerId

* chore(utils): move fast-utils to dev deps

* fix: allow autosharding nodes to get peers (#1785)

* fix: merge

* fix: build

* fix: failing tests from master merge

---------

Co-authored-by: Arseniy Klempner <arseniyk@status.im>
2024-01-19 20:42:52 +05:30
Arseniy Klempner
ae7bc3eeec
chore: bump all libp2p packages to latest version 2024-01-17 11:59:08 -08:00
Danish Arora
b99f828cfb
chore: minor refactoring for protocols (#1762)
* remove: unnecessary function

* remove: test

* update doc

* add: tests

* revert unintended change

* fix: ping return

* fix(tests): stub connections

* log warning instead of throwing an error
2024-01-11 17:25:47 +05:30
Danish Arora
0f7d63ef93
feat: Logger with log levels (#1672)
* setup a custom Logger with log level support

* refactor codebase for to use new Logger with log levels

* disallow usage of `debug` directly / only allow usage in/through custom Logger

* remove `debug` from logger
2023-10-20 16:36:47 +05:30
Danish Arora
124a29ebba
feat(static-sharding): filter peer connections per shards (#1626)
* add interface for `ShardInfo`

* enr: add deserialization logic & setup getters

* add sharding related utils

* utils: add shard<-> bytes conversion helpers

* pass `pubSubTopics` to `Waku`

* add `rs`/`rsv` details during discovery

* connection-manager: discard irrelevant peers

* add tests for static sharding - peer exchange

* update `ConnectionManager` tests to account for topic validity

* add js suffix to import

* address some comments

* move shardInfo encoding to ENR

* test: update for new API

* enr: add tests for serialisation & deserialisation

* address comment

* update test

* move getPeershardInfo to ConnectionManager and return ShardInfo instead of bytes

* update encoding and decoding relay shards to also factor for shards>64

* relay shard encoding decoding: use DataView and verbose spec tests

* improve tests for relay shard encoding decoding

* rm: only

* improve log message for unconfigured pubsub topic

* minor improvement

* fix: buffer <> Uint8array problems with shard decoding

* fix: test

* rm: only
2023-10-10 20:18:02 +05:30
Danish Arora
81a52a8097
feat!: set peer-exchange with default bootstrap (#1469)
* set peer-exchange with default bootstrap

* only initialise protocols with bootstrap peers

* update package

* update package-lock

* refactor `getPeers` while setting up a protocol

* move codecs to `@waku/interfaces`

* lightpush: send messages to multiple peers

* only use multiple peers for LP and Filter

* fix: ts warnings

* lightpush: tests pass

* update breaking changes for new API

* move codecs back into protocol files

* refactor: `getPeers()`

* rm: log as an arg

* add tsdoc for getPeers

* add import

* add prettier rule to eslint

* add: peer exchange to sdk as a dep

* fix eslint error

* add try catch

* revert unecessary diff

* revert unecessary diff

* fix imports

* convert relaycodecs to array

* remove: peerId as an arg for protocol methods

* keep peerId as an arg for peer-exchange

* remove: peerId from getPeers()

* lightpush: extract hardcoded numPeers as a constant

* return all peers if numPeers is 0 and increase readability for random peers

* refactor considering more than 1 bootstrap peers can exist

* use `getPeers`

* change arg for `getPeers` to object

* address comments

* refactor tests for new API

* lightpush: make constant the class variable

* use `maxBootstrapPeers` instead of `includeBootstrap`

* refactor protocols for new API

* add tests for `getPeers`

* skip getPeers test

* rm: only from test

* move tests to `base_protocol.spec.ts`

* break down `getPeers` into a `filter` method

* return all bootstrap peers if arg is 0

* refactor test without stubbing

* address comments

* update test title

* move `filterPeers` to a separate file

* address comments & add more test

* make test title more verbose

* address comments

* remove ProtocolOptions

* chore: refactor tests for new API

* add defaults for getPeers

* address comments

* rm unneeded comment

* address comment: add diversity of node tags to test

* address comments

* fix: imports
2023-09-07 13:15:49 +05:30
Danish Arora
b4f8216761
feat: pre-emptive stream creation for protocols (#1516)
* pass log as an arg to baseprotocol

* optimistically create and use streams for light protocols

* refactor BaseProtocol for readability

* use optimistic stream selection in protocols

* use a new stream for every request instead of reusing

* replenish streams correctly

* create StreamManager

* refactor for a single stream

* fix: listener binds

* declare streamManager as a class var isntead of extending

* remove stream destruction as it happens by default

* simplify logic & address comments

* fix: bind typo

* refactor for improvements

* fix typedoc

* rm: lock

* restructure StreamManager for readbility

* remove log as an arg

* use newStream as a facade in BaseProtoocl
2023-09-04 10:27:25 +05:30
Danish Arora
87717981eb
chore: upgrade libp2p and related deps (#1482)
* chore: update noise

* update: package.lock

* update: @chainsafe/libp2p-gossipsub

* rm unwanted libp2p interface deps & bump up libp2p

* refactor code for new deps

* update: new package.lock

* setup prettier, refactor eslint  and rm trailing commas

* update package.lock

* fix build

* import type for interface

* fix imports for merge

* update typedoc exports

* add: CustomEvent import

* use new libp2p interface

* add aegir as dev dep for tests
2023-08-16 20:18:13 +05:30
Danish Arora
1548a4f8e4
chore(peer-exchange): refactor/fix compliance test (#1478)
* use `peer:identify` instead of `peer:update`

* dial the peer after initialising peer-exchange
2023-08-16 11:14:33 +05:30
Sasha
51f9261a16
chore: update dependencies (#1460)
* update to latest prettier

* fix with prettier: added trailing comma

* remove deps from test package, add sinon types, fix type hack in a test, update esling prettier config

* update typescript eslint plugins

* update package-locks
2023-08-11 15:14:02 +02:00
Danish Arora
785df528fe
fix: improve connection manager error handling + edge cases (#1450)
* increase TTL on discovery classes

* refactor dialPeer to handle edge cases

* address comment
2023-08-02 13:49:48 +05:30
Danish Arora
6ce898d771
feat: enable event emission for peer discovery/connection in ConnectionManager (#1438)
* disable libp2p autodial

* improve logs for peer-exchange

* add a function to fetch discovered and connected peers by discovery

* connection-manager: introduce event emissions by discovery

* write a spec test for events

* minor code improvement for peer-exchange

* rm: comment

* rename peer event result interface

* switch to using libp2p EventEmitter

* rename variables for readability

* reset peer-exchange spec file

* address review

* test: minor refactor

* fix: failing test

* increase peer IDs to test against for attemptDial

* improve structuring
2023-07-26 22:51:55 +05:30
Sasha
420e6c698d
feat!: upgrade to libp2p@0.45 (#1400)
* upgrade libp2p version, partially update protocols, rename to IBaseProtocol

* complete transition for protocols

* complete transition of connection maanger

* finish sdk

* complete core

* complete relay

* complete peer-exchange

* complete dns-discovery

* add components field to Libp2p interface and use it in core

* add type hack for Libp2p creation:

* finish waku node test

* complete relay test

* complete peer exchange

* complete dns peer discovery test

* add missing dependency to relay

* fix new peer store integration

* improve initialization of pubsub

* add catch for missing peer

* update test and remove extra dependency

* prevent error throw

* fix edge case with peerStore

* fix peer exchange

* fix protocols used

* fix test with another evnet

* bump libp2p and interfaces

* add missing package

* fix peer-exchange problem

* prefer libp2p peerDiscovery for integration tests

* fix import

* increate timeout

* return test against Test fleet

* remove await for peer:update

* increase timeout

* add await for peerStore

* comment event for testing

* fix lint

* remove bind

* fix stub

* decouple to separate test case

* move back to explicit build

* remove only

* do not test event
2023-07-25 02:17:52 +02:00
Danish Arora
376bcf2a0a
chore: new eslint rules (#1416)
* add new eslint rules

* remove eslint scripts for build-utils

* refactor all files to fix breaking changes with new rules

* add a basic eslintrc for build-utils

* test

* fix more linting errors

* test

* update package.lock

* update stopping container

* update px node spec

* enable tests

* fix: conn manager events

* fix: peer exchange

* increase timeout on px test
2023-07-24 16:04:24 +05:30
dependabot[bot]
df8e68ff36
chore(deps)(deps): bump it-length-prefixed from 8.0.4 to 9.0.1
Bumps [it-length-prefixed](https://github.com/alanshaw/it-length-prefixed) from 8.0.4 to 9.0.1.
- [Release notes](https://github.com/alanshaw/it-length-prefixed/releases)
- [Changelog](https://github.com/alanshaw/it-length-prefixed/blob/master/CHANGELOG.md)
- [Commits](https://github.com/alanshaw/it-length-prefixed/compare/v8.0.4...v9.0.1)

---
updated-dependencies:
- dependency-name: it-length-prefixed
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-06 14:07:05 +10:00
Danish Arora
5b0c3c3cac
feat: compliance test for peer-exchange discovery (#1186)
* skip fleet test for CI

* add: compliance test

* fix: check CI

* fix: nwaku node name

* remove: setTimeout

* force typecasting instead of ts-ignore

* rm: only for the test

* increase readability
2023-03-23 16:06:51 +05:30
fryorcraken.eth
3a92355543
chore: code improvement
- Replace exception pattern with a result pattern as it better fits the
usage.
- merge few maps to prefer chaining to assign var after var.
- Make `isDefined` type helper a common util function
2023-03-21 08:59:25 +11:00
Danish Arora
a20b797c5b
chore: refactor peer-exchange according to nwaku 0.15.0 (#1193)
* merge with master: adhere acc to updated API

* bump go-waku to 0.5.2
2023-03-14 13:56:15 +05:30
fryorcraken.eth
5a42efcd8d
chore: rename: peer exchange peers are not bootstrap peers 2023-03-10 10:23:40 +11:00
fryorcraken.eth
6dbcde041a
chore!: directly convert from ENR to PeerInfo, remove unneeded utility 2023-03-10 10:23:39 +11:00
fryorcraken.eth
130c49b636
chore!: extract decoder code
Separation of concerns by moving decoding logic in new class.
2023-03-10 10:23:38 +11:00
fryorcraken.eth
952aadd7bb
fix(dns-discovery/peer-exchange): check if peer is already tagged
If we `continue` when the peer is already known by the `peerStore` then
the next step of checking the tags is useless (no tags if peer is not
known).

There are two ways around it:
1. either do nothing if peer is already in peer store
2. OR, do nothing if peer is in peer store and is already tagged

I opted for the second approach to ensure all peers are tagged properly.
2023-03-10 10:09:10 +11:00
fryorcraken.eth
1a9b13f902
chore: use BaseProtocol on WakuPeerExchange
Ref: https://github.com/waku-org/js-waku/pull/1137
2023-02-28 13:49:36 +11:00
fryorcraken.eth
7d29ed1d99
chore: move error throwing within selectPeerForProtocol
As all callers throw upon undefined result.
2023-02-27 09:48:30 +11:00
Danish Arora
1b41569023
fix: remove initialising peer-exchange while creating a node (#1158)
* remove peer-exchange from @waku/core
- also removes the manual test for peer-exchange (assumption is that the
only way to initialise peer-exchange is through libp2p's peerDiscovery and
not manually) (ref:
https://github.com/waku-org/js-waku/pull/1158#discussion_r1108055234)

 # Please enter the commit message for your changes. Lines
starting

* fix: build

* update interop test

* decrease test duration for px auto discovery

* rm: only for tests

* address comment
2023-02-17 13:27:37 +05:30
fryorcraken.eth
ab735908d3
chore: fix dependencies for @waku/peer-exchange` package 2023-02-09 19:31:12 +11:00
fryorcraken.eth
803ae7bd8e
chore!: bump libp2p dependencies 2023-02-09 19:30:59 +11:00
Danish Arora
b33306655a
chore: move libp2p and byte utils to a single utils package (#1149)
* move libp2p and byte utils to a single `utils`
package

* set package version to 0.0.1
2023-02-06 11:22:36 +05:30
Danish Arora
10b3898762
chore: move protocols CreateOptions into interfaces (#1145)
* move protocols `CreateOptions` into interfaces and
add possible TODO

* remove: createOptions for PeerExchange

* update test with new API
2023-02-02 08:02:06 +05:30
fryorcraken.eth
a5ff788eed
feat: codec as a property of the protocol implementations
For easy access from `@waku/core` without the need to depend on package
implementing the protocol.
2023-01-27 09:08:02 +11:00
Danish Arora
5fae073ebd
fix: edge case with peer-exchange (#1125)
* address comments

* rename to maxRetries

* use timeout instead of interval

* remove: only from test
2023-01-24 22:50:16 +05:30
Danish Arora
fcd500e045
fix: invoke peer exchange query immediately (#1115)
when the query is set in an interval, it first
gets invoked when the interval time is reached.
refactor: invoke it immmediately, and then set an
interval
2023-01-11 11:11:49 +05:30
Danish Arora
e0e8e655f8
chore: fix peer discovery peer-exchange (#1069)
* fix: discovery for peer-exchange

use the bootstrap node as a starter to send a
peer-exchange query to, and emit the response
peers received from it for further connection to
libp2p using the peer-discovery interface

* init: test for libp2p bootstrap/discovery for
peer-exchange

* temp-add: console.logs for easier debugging

* add: peer discovery test & rm: console.logs

* chore: rm  and redundant spec test

* add: interval for peer exchange queries
we set an interval to query a peer every 5 minutes
for peer exchange, and add new peers if found

* address: reviews
- add `type` for imports not using values
- better handling for peer-exchange query interval

* chore: fix tsc for peer-exchange
use node16 for module resolution

* chore: add extra exports to fix typedoc warnings
ref: https://github.com/TypeStrong/typedoc/issues/1739
2023-01-04 14:35:44 +05:30
fryorcraken.eth
ac4ff5a00a
chore: fix links in docs, treat doc warnings as errors 2022-12-21 16:47:00 +11:00
fryorcraken.eth
53af8994bd
feat: added I prefix to protocols interfaces
This will enable the remove of `Waku` prefix on all protocol
implementations, which is redundant due to the context.
2022-12-12 22:36:09 +11:00
Danish Arora
0ca7fd790c
feat!: implement peer exchange (#1027)
* wip -- yet to test

* update: draft

* wip

* support passing flags manually to nwaku node

* refactor peer-exchange test

* switch response from uint8array to ENR

* rm: unnecesary logs

* implement clas

* fix: for loop

* init-wip: directories

* setup: new package & fix circular deps

* bind a response handler

* wip: refactor & update test

* test logs

* wip code - debugging

* address: comments

* Update packages/core/src/lib/waku_peer_exchange/peer_discovery.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/core/src/lib/waku_peer_exchange/peer_discovery.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address: comments

* address: comments

* address: comments

* address: comments

* address: comments

* fix: test build

* refactor

* fix: build

* comply with API

* numPeers: use number instead of bigint

* fix: build

* Update packages/peer-exchange/package.json

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/waku_peer_exchange.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/waku_peer_exchange.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/waku_peer_exchange.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address: comments, add eslint config

* Update packages/peer-exchange/.eslintrc.cjs

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/index.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address comments

* test works with test fleet

* rm: only for px test => run all tests

* fix: tests

* reorder packages for build, and fix imports

* remove: px test doesnt work with local nodes

* chore: move proto into a separate package

* fix: proto dir

* fix: build

* fix: ci

* add: index for proto

* fix: ci

* Update packages/proto/package.json

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address comments

* chore: run failing test with higher timeout

* chore: run failing test with higher timeout

* fix: ci

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2022-12-07 11:35:30 +05:30