27 Commits

Author SHA1 Message Date
Sasha
c7682ea67c
feat!: re-architect connection manager (#2445)
* remove public pubsub field and redundant util

* add hangUp and improve dial operations, improve keepAliveManager and remove unused method, move utils and add tests

* improve public dial method to start keep alive checks

* move dial method

* implement discovery dialer

* implement discovery dialer with queue with tests

* add discovery dialer e2e tests, change local discovery log tag, update other tests

* remove comment

* add issue link, remove only

* implement shard reader component

* create evetns module, remove unused connection manager events and related tests

* implement network indicator

* implement connection limiter, change public API of connection manager, implement recovery strategy

* decouple keep alive maanger

* add connection manager js-doc

* refactor keep alive manager, cover with tests

* add tests for connection manager main facade

* add tests for connection limiter

* add e2e tests for connection manager modules
pass js-waku config during test node init
remove dns discovery for js-waku

* restructure dialing tests

* address last e2e tests

* address review

* add logging for main methods

* decouple pure dialer class, update network monitor with specific metrics

* remove console.log

* remove usage of protocols

* update sdk package tests

* add connect await promise

* add debug for e2e tests

* enable only packages tests

* use only one file

* revert debugging

* up interface for netwrok manager

* add logs

* add more logs

* add more logs

* add another logs

* remove .only

* remove log statements

* skip the test with follow up
2025-07-09 21:23:14 +02:00
Danish Arora
16328a3f11
feat: enable/disable discoveries without defaultBootstrap: true (#2363)
* feat: enable/disable discoveries without defaultBootstrap: true

* chore: rm .only

* chore: don't redefine default discoveries object

* chore: don't export default const from library

* chore: rename discoveriesEnabled to discovery

* fix: rename

* address comments

* chore: reset default

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>

* chore: address comments

* chore: update tests

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2025-05-30 16:27:00 +05:30
Sasha
163bea56c3
chore: skip flaky test due to nwaku bug (#2349)
* chore: skip flaky test due to nwaku bug

* return full test cases for lightpush
2025-04-11 18:30:28 +02:00
Sasha
3793e6f5c0
chore: update interop test suit for latest nwaku (0.35.1) (#2345)
* update Filter test suit, make service nodes connected to each other, remove single node Filter test suit, use 0.35 nwaku image

* update light push tests

* improve auto shard tests

* update static sharding test

* skip blocked tests

* fix test

* remove usage of pusubtopic with nwaku

* remove comment
2025-04-11 01:34:11 +02:00
Sasha
f199d92d60
feat: migrate to latest LightPush version (#2281)
* create retry manager

* update tests

* add retry manager tests, update peer manager

* fix start & merge with master

* return send to many logic

* add new error handling

* add sections to protocol errors

* fix check

* up test

* add waku.start in test

* fix check and test

* improve name
2025-02-25 22:40:03 +01:00
Sasha
87a918d9cc
chore: remove go-waku related code in tests (#2285)
* remove go-waku related code in tests

* remove not needed

* remove args level for go-waku
2025-02-25 17:56:55 +01:00
Danish Arora
a3fb1d7a5b
Revert "chore: upgrade nwaku to 0.34.0 and update tests suite for compatibili…" (#2273)
This reverts commit 26ab836900d907b0ca185e6ad5fb6b3b548350fb.
2025-02-21 23:06:45 +01:00
Danish Arora
26ab836900
chore: upgrade nwaku to 0.34.0 and update tests suite for compatibility (#2170)
* chore: upgrade nwaku to v0.33.1

* chore: upgrade to nwaku 0.34.0

* feat: connect nwaku nodes amongst each other over relay

* chore(lightpush): use multiple service nodes for lightpush (instead of just one)
- nwaku now expects >=1 nodes at least connected

* chore: all single-node lightpush requests should now be expected to fail

* chore: update sharding tests

* chore: update tests

* chore: improve Docker network config reliability

* chore: deduplicate ecies encrypted payloads

* chore: update to precise expects

* fix: return early if expect passes

* chore: lightpush 5 times instead of 30

* fix: non duplicacy should happen in application-specific scenario

* chore: update mocha config + fix epehermal tests

* chore: reinstall deps after rebase

* chore: attempt stability for test suite

* fix: store tests to now use multiple nodes, delete uneeded test

* fix: memory leak

* chore: switch while loop with timeout-promise

* chore: remove redundant nodes startup

* chore: add delays for nwaku setup
2025-02-17 19:21:33 +05:30
Sasha
62f93dc842
feat: improve peer manager and re-integrate to light push (#2191)
* up lock

* make ConnectionManager use ctor

* reform connection manager configurations

* remove log param from peerManager

* make PeerManager use only ConnectionManager, move getPeers to ConnectionManager, remove not needed code

* remove allPeers and connectedPeers from BaseProtocolCore, update tests, add getPeers for IWaku

* use only one peerManager from Waku object

* remove IBaseProtocolSDK and merge with PeerManager

* re-implement peerManager, remove ProtocolUseOptions

* remove not needed test, up lock

* update deps and lock

* remove old test for peerManager, fix check and spell

* rename to getConnectedPeers

* feat: improve filter subscriptions (#2193)

* add message cache to Filter

* remove WakuOptions and use only ProtocolCreateOptions

* move subscribe options to createLightNode Fitler protocol options

* rename SubscriptionManager to Subscription

* rename to CreateNodeOptions

* add warning

* feat: introduce subscription manager (#2202)

* feat: inroduce subscription manager

* fix: make pipeline succeed (#2238)

* fix test

* use hardcoded value

* update playwright

* fix test:browser

* up lock

* make peer retrieval probabilistic

* add comments

* up lightpush tests

* add tests for peer_manager, improve folder structure

* create named files for protocols

* create named files, simplify project structure

* remove only
2025-01-31 00:16:00 +01:00
Sasha
75fcca4cd9
feat: replace waitForRemotePeers() with waku.waitForPeer() method (#2161)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* add I prefix to Waku interface

* implement waku.connect method

* add docs to IWaku interface

* remove export and usage of waitForRemotePeer

* move wait for remote peer related to Realy out of @waku/sdk

* change tests to use new API

* fix linting

* update size limit

* rename .connect to .waitForPeer

* export waitForRemotePeer and mark as deprecated

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer (#2163)

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer

* add waitForRemote UTs

* remove junk

* feat: expose peerId and protocols from WakuNode (#2166)

* chore: expose peerId and protocols from WakuNode

* remove unused method

* move to private method

* rename to waitForPeers

* up test
2024-10-09 00:43:34 +02:00
Sasha
d37e0245cf
feat: confirm metadata and protocols needed in waitForRemotePeer (#2160)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* imrpove iteration for existing connections

* address protocol adverisement in CI

* add protocols needed

* add missing protocols

* make lightpush and filter default for tests

* up
2024-10-05 01:49:54 +02:00
Sasha
1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Danish Arora
9d9a696024
chore: mandate .js extensions for imports (#2128)
* feat: add eslint rule

* chore: update codebase import extensions
2024-09-13 14:18:29 +05:30
Sasha
5cfe9327b6
chore: remove relay from sdk package (#2040)
* remove full node

* remove relay from sdk

* comment node counter check

* try using logline

* up comment
2024-08-31 15:18:51 +02:00
Sasha
71384dfdfd
chore: change amount of used peers to 2 (#2111)
* chore: change amount of used peers to 2

* up

* up

* up tests
2024-08-27 21:53:16 +02:00
Danish Arora
5ce36c8f18
feat!: deprecate named pubsub topics and use static/auto sharding (#2097)
* feat: deprecate named sharding & protocols adhere
simplify network config type, all protocols use pubsub topic internally

* chore: update tests

* tests: rm application info

* chore: use static sharding and auto sharding terminologies

* chore: update docs for network config

* chore: update interfaces

* tests: update tests error message

* chore: remove `ShardingParams` type and fix test
2024-08-13 05:23:20 +05:30
Danish Arora
d464af3645
feat: node and protocols health (#2080)
* feat: introduce HealthManager

* feat: make health accessible on Waku object

* feat: update health from protocols

* chore: add access modifiers to healthmanager

* feat: use a HealthManager singleton

* chore: add tests for Filter, LightPush and Store

* feat: add overall node health

* chore: update protocol health to consider Store protocol

* chore: setup generic test utils instead of using filter utils

* tests: add a health status matrix check from 0-3

* chore: increase timeout for failing tests in CI
tests pass locally without an increased timeout, but fail in CI

* chore: move name inference to HealthManager

* tests: abstract away node creation and teardown utils

* fix: import
2024-07-27 18:27:54 +05:30
Danish Arora
4eb06c64eb
feat(filter)!: return error codes instead of throwing errors (#1971)
* move protocol result type to interfaces

* chore: update type names for verbosity

* feat(filter-core): convert error throws to return types

* chore: update types & imports

* update Filter API

* chore: update createSubscription

* chore: update imports & rename

* chore: update all tests

* chore: resolve conflicts & merge (2/n)

* chore: resolve conflicts & merge (3/n)

* chore: resolve conflicts & merge (4/n)

* chore: resolve conflicts & merge (5/n)

* chore: resolve conflicts & merge (6/n)

* chore: use idiomatic approach

* chore: fix tests

* chore: address comments

* chore: fix test

* rm: only
2024-05-09 16:51:08 +05:30
Sasha
f3627c46a4
feat!: use ShardingParams on subscriptions, make Decoder/Encoder auto sharding friendly by default (#1958)
* fix: use pubsubTopic from current ones if not set

* fix: improve type on dial method

* enforce same pubusb on filter.subscribe, make content topic to pubsub mapping default for decoder / encoder

* fix mapping problem

* update tests

* add error handling

* fix typo

* up lock

* rm lock

* up lock

* remove only

* fix content topic

* fix ephemeral test

* fix filter unsubscribe test

* up utils

* fix subscribe test

* up interfaces and filter api

* remove only

* up ping test

* fix subscribe test

* fix push test

* fix lightPush

* fix multiple pubsub

* remove only, fix subscribe filter test

* remove only

* fix cluster ID selection and named sharding subscription test

* fix unsubscribe test

* fix light push test

* fix light push test

* fix push test

* fix relay publish

* create runNode and fix relay tests

* generalize runNodes, fix some tests

* fix store tests

* fix toAsyncIterator tests

* remove only

* fix lightPush

* use generics

* try fix test

* run failing tests

* remove only

* address failed tests, remove DefaultPubsubTopic dependency in some tests
2024-04-28 11:15:17 +02:00
fbarbu15
7ee02faf48
chore: use nwaku:v0.27.0 and adjust tests for it (#1975)
* use nwaku:v0.27.0 and adjust tests for it

* fix sharding tests
2024-04-24 21:30:55 +02:00
Florin Barbu
f4c7c02f83
test: sharding tests refactor (#1883)
* sharding tests refactor

* small fixes

* adjust clusterID based on version

* fix typo

* fix dispatchEvent test

* sharding unit tests

* port adjustment

* update unit tests

* fix 1902

* adjust content topic tests

* adjust metdata tests

* small adjustments

* fix

* update resolveAutoshardingCluster version

* skip autosharding tests for nwaku < 0.27.0

* skip autosharding tests for nwaku < 0.27.0
2024-03-18 13:40:08 +02:00
Florin Barbu
df8c0d79f7
test: peer exchange tests (#1859)
* peer exchange tests

* adjust after test with nwaku 25
2024-02-27 09:54:34 +02:00
Florin Barbu
de5be4413b
chore: use graceful timeout mechanism (#1841)
* use graceful timeout mechanism

* set max timeout for all hooks

* small fix

* small fix

* use MOCHA_HOOK_MAX_TIMEOUT as default timeoutDuration

* force retry even when the hook fails

* use custom hooks

* fix global timeout problem

* fix unwanted change

* fix enr issue

* force retry on before error as well
2024-02-15 08:50:03 +02:00
Danish Arora
7affbe222d
feat: lightpush & filter send requests to multiple peers (#1779)
* feat: lightpush & filter send requests to multiple peers

* build message-hash before core

* chore: restructure folder heirrarchy

* fix: imports

* chore: move @waku/core to dev deps

* feat: create a new `ServiceNodes` wrapper class to encapsulate service node and message collector with redundancy accounted for

* chore(filter): move tests against single service node to a subdir

* feat: support relay, add strict checking, add tests

* fix(filter): handle errors

* chore(tests): add tests for ping

* add tests for push

* chore: abstract redundancy

* feat: add unsubscribe tests

* fix: tests

* add lightpush tests

* fix: imports

* fix: merge & add warning

* merge: master

* fix: breaking tests with master

* address comments

* make num peers configurable

* fix: typo
2024-01-24 18:24:03 +05:30
Florin Barbu
13d3d70609
chore: increase max sub topic size to 100 (#1791)
* increase max sub topic size to 100

* make tests run fine both ways

* fix:import error

* update nwaku and gowaku master images

* prepare for pr
2024-01-18 21:22:51 +05:30
Arseniy Klempner
ae7bc3eeec
chore: bump all libp2p packages to latest version 2024-01-17 11:59:08 -08:00
Danish Arora
2e6d9836bf
chore(tests): restructure & cleanup (#1796)
* chore: restructure folder heirrarchy

* fix: imports
2024-01-18 00:26:31 +05:30