391 Commits

Author SHA1 Message Date
Sasha
f55db3eb4b
feat: shard retrieval for store and store peers selection (#2417)
* feat: implement shard retrieval for store and improve set store peers usage

* remove log

* remove only, improve condition

* implement smarter way to retrieve peers

* up tests

* update mock

* address nits, add target to eslint, revert to es2022
2025-06-23 10:01:54 +02:00
Sasha
fcc6496fef
chore!: upgrade libp2p, nodejs and typescript (#2401)
* remove react native package

* upgrade nodejs

* upgrade libp2p

* upgrade typescript, use 22 node

* up lock

* fix node compatibility

* upgrade playwright

* upgrade proto

* update tests
2025-06-22 17:41:23 +02:00
Danish Arora
fecc026dd7
fix: DIRVER -> DRIVER typo in nwaku (#2406)
* fix: DIRVER -> DRIVER nwaku

* chore: empty commit

* chore: access CI/test env vars and conditionally check test

* fix: undefined nwaku version

* chore: improve splitting

* simplify, test error string only for new versions

---------

Co-authored-by: Sasha <oleksandr@status.im>
2025-06-20 15:34:51 +02:00
Sasha
7c8d1073b0
chore!: remove IBaseProtocol and improve interface on PeerExchange (#2422)
* remove IBaseProtocol

* fix references, interfaces and integration

* fix ci

* up mock

* up lock

* add mock for local storage

* add missing prop, fix tests

* up lock
2025-06-20 12:53:42 +02:00
Arseniy Klempner
f649f59e64
fix: rebase package lock 2025-06-03 11:24:37 -07:00
Arseniy Klempner
5f63cb5bfb
fix: remove unnecessary comments from store validation
- Remove inline comments that explain obvious code behavior
- Keep only the reference URL comment which provides useful context
- Address review feedback about excessive commenting
2025-06-03 09:05:54 -07:00
Arseniy Klempner
2d92191029
fix: address review feedback for message hash tests
- Remove all console.log statements (8 instances)
- Update type usage from DecodedMessage[] to IDecodedMessage[]
- Replace assert.equal() with expect().to.equal() for consistency
- Import expect from chai alongside existing imports

These changes address all feedback from PR review without altering test logic.
2025-06-03 09:05:54 -07:00
Arseniy Klempner
9f7a15dfb1
fix(store): update store query validation logic to support msg hash queries 2025-06-03 09:05:54 -07:00
Sasha
1905558753
feat!: re-work messaging parts and sharding (#2399)
* remove message-hash package, move it to core

* add js-doc to message hash

* up

* address type changes

* fix lint
2025-06-03 11:08:02 +02:00
Danish Arora
16328a3f11
feat: enable/disable discoveries without defaultBootstrap: true (#2363)
* feat: enable/disable discoveries without defaultBootstrap: true

* chore: rm .only

* chore: don't redefine default discoveries object

* chore: don't export default const from library

* chore: rename discoveriesEnabled to discovery

* fix: rename

* address comments

* chore: reset default

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>

* chore: address comments

* chore: update tests

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2025-05-30 16:27:00 +05:30
Sasha
a4dfd3455c
feat: event based approach to Filter (#2300)
* create new filter api

* implement await on main methods on new Filter

* add info logs in new filter

* add logs to subscription impl

* remove lint supress

* add unit tests

* introduce E2E tests

* update e2e tests and add case for testing filter recovery after nwaku nodes replacement

* add new test cases for max limits and enable decoders as array on new filter

* fix edge case testing, correct test cases

* skip test

* update error message

* up text

* up text

* fix lint

* implement unsubscribeAll

* add js-dock to new filter

* add cspell

* implement TTL set for message history
2025-05-28 00:44:44 +02:00
Sasha
fd2f5b7b8d
chore: up readme (#2367) 2025-04-18 20:44:22 +02:00
Sasha
3038c48917
feat: introduce createDecoder and createEncoder on IWaku (#2352)
* feat: introduce createDecoder and createEncoder on IWaku

* add tests, refactor

* fix type
2025-04-14 10:46:47 +02:00
Sasha
163bea56c3
chore: skip flaky test due to nwaku bug (#2349)
* chore: skip flaky test due to nwaku bug

* return full test cases for lightpush
2025-04-11 18:30:28 +02:00
Sasha
3793e6f5c0
chore: update interop test suit for latest nwaku (0.35.1) (#2345)
* update Filter test suit, make service nodes connected to each other, remove single node Filter test suit, use 0.35 nwaku image

* update light push tests

* improve auto shard tests

* update static sharding test

* skip blocked tests

* fix test

* remove usage of pusubtopic with nwaku

* remove comment
2025-04-11 01:34:11 +02:00
fryorcraken
d27db21ba5
fix: do not use waku test fleet as default bootstrap (#2312)
The waku test fleet is unstable and should not be used as boostrap.

The issue is that DNS Discovery looks for "2" nodes and stop there. Meaning that if those "discovered" nodes are not reachable (ie, coming from test suite). js-waku does not connect.
2025-03-20 16:32:27 +01:00
fryorcraken
edfb56243d
doc: build first to avoid not found deps issues (#2307)
* doc: build first to avoid not found deps issues

* fix: Remove XMLHttpRequest usage

XMLHttpRequest API is deprecated and not available in browser extensions.

Replace the culprit dependency with a more modern one.

Some options are removed. The assumption is that nobody uses them. It can always be added if a developer wants the flexibility.

* test: simplify test

The test focus on testing DNS Discovery, there is no need to also have a nwaku local node for bootstrap.

Bootstrap on nwaku local node is used in many other tests.
2025-03-19 09:43:41 +01:00
Sasha
f199d92d60
feat: migrate to latest LightPush version (#2281)
* create retry manager

* update tests

* add retry manager tests, update peer manager

* fix start & merge with master

* return send to many logic

* add new error handling

* add sections to protocol errors

* fix check

* up test

* add waku.start in test

* fix check and test

* improve name
2025-02-25 22:40:03 +01:00
Sasha
87a918d9cc
chore: remove go-waku related code in tests (#2285)
* remove go-waku related code in tests

* remove not needed

* remove args level for go-waku
2025-02-25 17:56:55 +01:00
Danish Arora
a3fb1d7a5b
Revert "chore: upgrade nwaku to 0.34.0 and update tests suite for compatibili…" (#2273)
This reverts commit 26ab836900d907b0ca185e6ad5fb6b3b548350fb.
2025-02-21 23:06:45 +01:00
Danish Arora
26ab836900
chore: upgrade nwaku to 0.34.0 and update tests suite for compatibility (#2170)
* chore: upgrade nwaku to v0.33.1

* chore: upgrade to nwaku 0.34.0

* feat: connect nwaku nodes amongst each other over relay

* chore(lightpush): use multiple service nodes for lightpush (instead of just one)
- nwaku now expects >=1 nodes at least connected

* chore: all single-node lightpush requests should now be expected to fail

* chore: update sharding tests

* chore: update tests

* chore: improve Docker network config reliability

* chore: deduplicate ecies encrypted payloads

* chore: update to precise expects

* fix: return early if expect passes

* chore: lightpush 5 times instead of 30

* fix: non duplicacy should happen in application-specific scenario

* chore: update mocha config + fix epehermal tests

* chore: reinstall deps after rebase

* chore: attempt stability for test suite

* fix: store tests to now use multiple nodes, delete uneeded test

* fix: memory leak

* chore: switch while loop with timeout-promise

* chore: remove redundant nodes startup

* chore: add delays for nwaku setup
2025-02-17 19:21:33 +05:30
Danish Arora
0a0a92bccb
feat: @waku/rln (#2244)
* chore: setup rln as a new package

* chore: migrate src

* fix: wasm loading, tests, config

* chore: fix Karma CI

* fix: bundler

* chore: copy dist resources

* chore(rln): enable all tests

* chore: increase karma timeouts
2025-02-11 15:28:00 +05:30
Sasha
3136f3a704
feat: add HealthIndicator with simplified logic and testing (#2251)
* implement HealthIndicator

* up libp2p interface version

* up lock

* remove unused tests

* expose HealthIndicator from Waku

* update test, add start and stop

* fix error handling
2025-02-05 13:24:50 +01:00
fbarbu15
2a7f4b6e86
chore: add back allure (#2247) 2025-02-03 13:59:55 +05:30
Sasha
62f93dc842
feat: improve peer manager and re-integrate to light push (#2191)
* up lock

* make ConnectionManager use ctor

* reform connection manager configurations

* remove log param from peerManager

* make PeerManager use only ConnectionManager, move getPeers to ConnectionManager, remove not needed code

* remove allPeers and connectedPeers from BaseProtocolCore, update tests, add getPeers for IWaku

* use only one peerManager from Waku object

* remove IBaseProtocolSDK and merge with PeerManager

* re-implement peerManager, remove ProtocolUseOptions

* remove not needed test, up lock

* update deps and lock

* remove old test for peerManager, fix check and spell

* rename to getConnectedPeers

* feat: improve filter subscriptions (#2193)

* add message cache to Filter

* remove WakuOptions and use only ProtocolCreateOptions

* move subscribe options to createLightNode Fitler protocol options

* rename SubscriptionManager to Subscription

* rename to CreateNodeOptions

* add warning

* feat: introduce subscription manager (#2202)

* feat: inroduce subscription manager

* fix: make pipeline succeed (#2238)

* fix test

* use hardcoded value

* update playwright

* fix test:browser

* up lock

* make peer retrieval probabilistic

* add comments

* up lightpush tests

* add tests for peer_manager, improve folder structure

* create named files for protocols

* create named files, simplify project structure

* remove only
2025-01-31 00:16:00 +01:00
Ivan FB
d706fe146c
replace pubsub topic param with clusterid plus shard (#2239)
We are in the process of deprecating pubsub-topic param from nim-waku.
The pubsub-topic shouldn't be used to start nim-waku and instead, a
combination of 1 cluster-id plus n shard should be given. This aims to
start using the new added param: shard.
2025-01-30 18:16:07 +05:30
Danish Arora
f3aed7074b
chore: switch out of allure reporters (#2222)
* chore: switch out of allure

* chore: fix config

* fix: test file patterns
2025-01-27 21:14:34 +05:30
Danish Arora
7c0ce7b2ec
chore: upgrade to libp2p v2 (#2143)
* chore: upgrade libp2p to v2 and related deps

* chore: fix ENR

* chore(core): remove CustomEvent polyfill import

* chore: `peer-id-factory` has been removed

* chore(discovery): fix local-cache & remove CustomEvent imports

* chore(sdk): update config

* chore(tests): update tests without peer-id-factory

* fix: spec tests

* chore: fix test

* chore: upgrade dataset-core

* chore: upgrade libp2p and stale references

* chore: upgrade playwright

* chore: rm console log

* fix: lock
2024-10-21 16:43:24 +05:30
Sasha
4049123f14
feat!: lighten retry logic for LightPush (#2182)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* skip tests
2024-10-17 00:49:24 +02:00
Danish Arora
b2efce5ec2
feat(filter): enhancing protocol peer management with mutex locks (#2137)
* chore: improvements

* chore: add logs for subscription maintenance

* chore: update logging

* chore: trimming down BaseProtocolCore

* chore: track peers in a hashmap instead of array

* chore: peer mgmt responds to conenction/disconnection and improve logging

* feat: add mutex locks to tackle race conditions over shared state

* fix: build

* chore: some mutex lock-release improvements

* feat: peer manager

* chore: rm tests for remove internal util

* chore: update HealthManager updates

* chore: update tests

* rm: only

* fix: hasPeers management

* chore: add modularity to getting connected peers

* chore: improve logs & add debug

* chore: renewal doesnt disconnect, only removes

* chore: await for sequential operations

* chore: add TODO

* chore: minor improvements

* chore: fix rebase

* chore: update playright

* chore: remove additional arg

* chore: update interafce

* feat(peer-manager): unit tests

* chore: improve hasPeers()

* chore: update lockfile

* feat: Filter reacts to peer:disconnect event, add tests

* chore: fix lock

* chore: update playright

* chore: update protocol health for lightpush

* chore: remove .only

* chore: address comments and improvements

* fix: tsconfig
2024-10-10 23:47:12 +02:00
Sasha
75fcca4cd9
feat: replace waitForRemotePeers() with waku.waitForPeer() method (#2161)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* add I prefix to Waku interface

* implement waku.connect method

* add docs to IWaku interface

* remove export and usage of waitForRemotePeer

* move wait for remote peer related to Realy out of @waku/sdk

* change tests to use new API

* fix linting

* update size limit

* rename .connect to .waitForPeer

* export waitForRemotePeer and mark as deprecated

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer (#2163)

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer

* add waitForRemote UTs

* remove junk

* feat: expose peerId and protocols from WakuNode (#2166)

* chore: expose peerId and protocols from WakuNode

* remove unused method

* move to private method

* rename to waitForPeers

* up test
2024-10-09 00:43:34 +02:00
Sasha
d37e0245cf
feat: confirm metadata and protocols needed in waitForRemotePeer (#2160)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* imrpove iteration for existing connections

* address protocol adverisement in CI

* add protocols needed

* add missing protocols

* make lightpush and filter default for tests

* up
2024-10-05 01:49:54 +02:00
Sasha
2be0e81a0a
chore: rename IProtocolSDK interfaces to IProtocol naming convention (#2159)
* chore: rename IProtocolSDK interfaces to IProtocol
2024-10-04 13:53:54 +02:00
Sasha
1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Sasha
b93134a517
fix: peer renewal connection drop & stream management (#2145)
* fix: peer renewal connection drop

* fix stream manager

* fix over iteration during stream creation

* remove timeout and use only open peers

* add logs

* refactor code, add tests

* debug test

* up debug

* remove debug, supress check for timestamps

* remove only

* add more debug

* remove debug

* remove check for timestamps
2024-10-01 12:54:55 +02:00
Danish Arora
7a6247cb70
feat(lightpush): introduce ReliabilityMonitor and allow send retries (#2130)
* chore: restructure reliabiltiy monitors

* feat: setup sender monitor

* chore: update tests

* chore: minor fixes

* chore: comment for doc
2024-09-17 11:34:59 +05:30
Danish Arora
7ad1d321ca
feat(filter): reliability monitor as a separate class to handle reliability logic (#2117)
* chore: move SubscriptionManager to a separate file

* feat: introduce ReliabilityMonitor

* fix: peer data updates

* fix: logical error when returning includesMessage

* chore: handle edge case

* chore: move ping failures handling inside monitor

* chore: move renewal logic to monitor

* chore: improve structuring

* chore: update logger

* chore: readd connectionListener() logic from merge

* chore: minor fixes

* chore: improve

* chore: setup destruction of ReliabilityMonitors

* fix: condition for ping failure

* fix: setters

* chore: handle race condition & fix test
2024-09-13 14:57:29 +05:30
Danish Arora
9d9a696024
chore: mandate .js extensions for imports (#2128)
* feat: add eslint rule

* chore: update codebase import extensions
2024-09-13 14:18:29 +05:30
Sasha
5cfe9327b6
chore: remove relay from sdk package (#2040)
* remove full node

* remove relay from sdk

* comment node counter check

* try using logline

* up comment
2024-08-31 15:18:51 +02:00
Sasha
eadb85ab83
feat: offline state recovery for Filter subscription (#2049)
* up

* fix window reference

* add tests

* up

* add e2e renew test

* address comments

* remove unused

* add test

* try

* remove only

* up test

* up

* remove only

* add tmp logs, use before/after hooks

* up

* fix check

* remove only

* fix test

* up
2024-08-28 18:00:18 +02:00
Sasha
71384dfdfd
chore: change amount of used peers to 2 (#2111)
* chore: change amount of used peers to 2

* up

* up

* up tests
2024-08-27 21:53:16 +02:00
Danish Arora
69b6612ec9
chore: upgrade node to LTS 20 (#2110)
* chore: upgrade nodejs to new LTS 20

* chore: update other references
2024-08-23 15:57:20 +05:30
Danish Arora
409642d7d3
chore: upgrade libp2p and related packages (#2084)
* chore: upgrade libp2p and related packages

* fix: public key on peer id can now be undefined

* chore: upgrade remainder packages

* chore: fix tests

* chore: address comments

* chore: upgrade playwright and proto
2024-08-14 04:23:04 +05:30
Danish Arora
5ce36c8f18
feat!: deprecate named pubsub topics and use static/auto sharding (#2097)
* feat: deprecate named sharding & protocols adhere
simplify network config type, all protocols use pubsub topic internally

* chore: update tests

* tests: rm application info

* chore: use static sharding and auto sharding terminologies

* chore: update docs for network config

* chore: update interfaces

* tests: update tests error message

* chore: remove `ShardingParams` type and fix test
2024-08-13 05:23:20 +05:30
Danish Arora
86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Danish Arora
fdd9dc44a4
feat(filter)!: new simpler filter API (#2092)
* chore: rename IReceiver subscribe

* feat!: new `subscribe() API that only takes in decoders and callback

* chore: `to_async_iterator` uses new function name

* chore: make `createSubscription` private, and shorten error handling

* chore: update subscribe return type

* tests: use new API

* fix: tests
2024-08-05 15:52:58 +05:30
Danish Arora
33a9172bbb
chore: upgrade nwaku to 0.31 (#2091)
* chore: upgrade nwaku to 0.31

* tests: fix tests by removing reliance on named sharding
2024-08-01 14:33:42 +05:30
Danish Arora
d464af3645
feat: node and protocols health (#2080)
* feat: introduce HealthManager

* feat: make health accessible on Waku object

* feat: update health from protocols

* chore: add access modifiers to healthmanager

* feat: use a HealthManager singleton

* chore: add tests for Filter, LightPush and Store

* feat: add overall node health

* chore: update protocol health to consider Store protocol

* chore: setup generic test utils instead of using filter utils

* tests: add a health status matrix check from 0-3

* chore: increase timeout for failing tests in CI
tests pass locally without an increased timeout, but fail in CI

* chore: move name inference to HealthManager

* tests: abstract away node creation and teardown utils

* fix: import
2024-07-27 18:27:54 +05:30
Danish Arora
defe41bb9a
feat(peer-exchange): support continuous peer information updates (#2088)
* feat(peer-exchange): update peer info if new is found

* chore: move diff checking logic to a private function

* chore: add tests

* chore: increase verbosity for mulltiaddr

* chore: use merge for metadata nad patch for multiaddrs

* chore: use peerId from peerInfo

* chore: remove unused import
2024-07-26 17:14:55 +05:30
gabrielmer
08fc2d133a
chore: changing default pubsub topic to its static sharding version (#2083)
* changing default pubsub topic to its static sharding version

* keeping RFC's Waku Message test vectors

* reverting change in changelog

* setting pubsub topic when creating nwaku node

* adding shardInfo to runMultipleNodes call

* adding shardInfo to runMultipleNodes call in lightpush tests

* add pubsub topics to nwaku.start

* get rid of it.only that remained

* fixing compliance tests

* setting clusterId to 0

* removing unnecessary fix

* adding shardInfo when creating nodes

* fixing wait for remote peer tests

* fixing peer exchange test

* refactor

* removing unnecessary variable

* feat: create default shard info, update tests (#2085)

* feat: create default shard info, update tests

* add link

* fix tests

* remoe only

* up tests

* up test

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2024-07-26 00:21:52 +02:00