806 Commits

Author SHA1 Message Date
Arseniy Klempner
408be95a13
feat(sds): add retrieval hint to causal history 2025-04-22 07:45:05 -07:00
Arseniy Klempner
6b4848c853
feat(sds): adds ephemeral messages, delivered message callback and event 2025-04-22 07:16:14 -07:00
Sasha
fd2f5b7b8d
chore: up readme (#2367) 2025-04-18 20:44:22 +02:00
Sasha
551238006c
fix: health indicator properly handle peer:identify event (#2369) 2025-04-18 20:11:09 +02:00
Sasha
3038c48917
feat: introduce createDecoder and createEncoder on IWaku (#2352)
* feat: introduce createDecoder and createEncoder on IWaku

* add tests, refactor

* fix type
2025-04-14 10:46:47 +02:00
Sasha
163bea56c3
chore: skip flaky test due to nwaku bug (#2349)
* chore: skip flaky test due to nwaku bug

* return full test cases for lightpush
2025-04-11 18:30:28 +02:00
Sasha
3793e6f5c0
chore: update interop test suit for latest nwaku (0.35.1) (#2345)
* update Filter test suit, make service nodes connected to each other, remove single node Filter test suit, use 0.35 nwaku image

* update light push tests

* improve auto shard tests

* update static sharding test

* skip blocked tests

* fix test

* remove usage of pusubtopic with nwaku

* remove comment
2025-04-11 01:34:11 +02:00
Danish Arora
28f28d0d36
fix(rln): update types to match nwaku (#2339)
* chore: update types to match nwaku

* chore: add userMessageLimit // not in spec
2025-04-10 14:42:57 +05:30
Danish Arora
4adf8706c3
feat(rln): create CredentialsManager without Zerokit (#2295)
* chore: update ABI

* chore: update contract address and chain ID to Linea Sepolia

* chore: improve error handling

* fix: bigint conversion

* chore: update tests

* chore: clean comments

* chore: export keystore types

* chore: update README with contract address

* tests: add reusable mock functions

* chore: LINEA_CONTRACT instead of SEPOLIA_CONTRACT

* chore: add linea to cspell

* chore: add rateLimit to MembershipInfo

* fix: determine start options

* feat: RLNLight

* chore: fix

* chore: export

* fix: returns for ratelimit

* chore: big number conversions

* chore: rename RLNLight to CredentialsManager, add logs

* chore: setup and use rln_base_contract

* chore: use CredentialsManager for rln.ts

* chore: public methods written above private methods

* fix: rate limit getter

* chore: simplify getters/setters

* chore: insert empty line
2025-04-07 16:04:06 +05:30
Danish Arora
a8ff776962
chore(rln): update ABI and contract address to Linea Sepolia, enhancements (#2294)
* chore: update ABI

* chore: update contract address and chain ID to Linea Sepolia

* chore: improve error handling

* fix: bigint conversion

* chore: update tests

* chore: clean comments

* chore: export keystore types

* chore: update README with contract address

* tests: add reusable mock functions

* chore: LINEA_CONTRACT instead of SEPOLIA_CONTRACT

* chore: add linea to cspell

* chore: add rateLimit to MembershipInfo

* fix(tests): rate limit additions

* chore: update gitignore
2025-03-31 14:32:29 +05:30
Sasha
ea6daae927
chore: release master (#2311) 2025-03-24 14:41:33 +01:00
Sasha
1ee77c0423
fix: downgraid chainsafe (#2316)
* fix: downgraid chainsafe

* up lock

* up lock

* bump playwright

* add override section
2025-03-24 13:07:29 +01:00
fryorcraken
d27db21ba5
fix: do not use waku test fleet as default bootstrap (#2312)
The waku test fleet is unstable and should not be used as boostrap.

The issue is that DNS Discovery looks for "2" nodes and stop there. Meaning that if those "discovered" nodes are not reachable (ie, coming from test suite). js-waku does not connect.
2025-03-20 16:32:27 +01:00
Sasha
e0b8bd8c9c
chore: update release please and add packages (#2305) 2025-03-20 00:40:34 +01:00
fryorcraken
edfb56243d
doc: build first to avoid not found deps issues (#2307)
* doc: build first to avoid not found deps issues

* fix: Remove XMLHttpRequest usage

XMLHttpRequest API is deprecated and not available in browser extensions.

Replace the culprit dependency with a more modern one.

Some options are removed. The assumption is that nobody uses them. It can always be added if a developer wants the flexibility.

* test: simplify test

The test focus on testing DNS Discovery, there is no need to also have a nwaku local node for bootstrap.

Bootstrap on nwaku local node is used in many other tests.
2025-03-19 09:43:41 +01:00
Arseniy Klempner
13ae5d4f73
feat(sds): send and receive sync messages 2025-03-13 11:29:55 -07:00
Arseniy Klempner
f7666a6588
feat(sds): adds logic to sweep incoming and outgoing buffers 2025-03-03 22:09:44 -08:00
Danish Arora
6fc6bf3916
feat(rln): migrate from v1 to v2, rate limiting, memberships, test coverage (#2262)
* chore: update ABIs and deployed address

* chore: remove storage contract references

* feat: upgrading adapter to basic rlnv2

* feat: rate limit

* chore: upgrade packages revert

* fix: tests

* chore: remove uneeded file

* feat(rln): implement RLNv2 rate limiting and membership states

- Add rate limit validation and handling in proof generation/verification
- Implement membership lifecycle state management (Active/GracePeriod/Expired)
- Add new membership management methods:
  - getMembershipInfo
  - extendMembership
  - eraseMembership
  - registerMembership
- Update proof verification to include rate limit checks
- Refactor message serialization to include rate limit data

Breaking changes:
- verifyWithRoots now takes roots as array instead of spread parameters
- Proof verification methods now accept optional rateLimit parameter

* fix: typo

* chore: add to cspell

* chore: reduce diff

* chore: simplify subdir for abi

* chore: address comments

* chore: simplify access to variables

* chore: address comments

* chore: simplify constants

* chore: add error handling

* chore: change rln v2 references to rln

* fix: check
2025-03-03 18:14:06 +05:30
Sasha
09108d9284
feat: enable auto start upon node creation (#2291)
* feat: enable auto start upon node creation

* update the state
2025-02-27 14:00:28 +01:00
Sasha
f199d92d60
feat: migrate to latest LightPush version (#2281)
* create retry manager

* update tests

* add retry manager tests, update peer manager

* fix start & merge with master

* return send to many logic

* add new error handling

* add sections to protocol errors

* fix check

* up test

* add waku.start in test

* fix check and test

* improve name
2025-02-25 22:40:03 +01:00
Sasha
0ede57f387
chore: simplify protocol creation (#2283)
* simplify protocol creation

* up relay

* fix check

* up size limit config
2025-02-25 18:41:32 +01:00
Sasha
87a918d9cc
chore: remove go-waku related code in tests (#2285)
* remove go-waku related code in tests

* remove not needed

* remove args level for go-waku
2025-02-25 17:56:55 +01:00
Arseniy Klempner
389ca4062e
feat(sds): add message channel with buffers and send/receive logic
This commit creates the class for an SDS message channel, including
buffers for outgoing and incoming messages. Adds logic for sending
messages, receiving messages, delivering messages, and reviewing
acknowledgement status of messages. Also adds byte serialization
for bloom filters.
2025-02-21 14:12:29 -08:00
Danish Arora
a3fb1d7a5b
Revert "chore: upgrade nwaku to 0.34.0 and update tests suite for compatibili…" (#2273)
This reverts commit 26ab836900d907b0ca185e6ad5fb6b3b548350fb.
2025-02-21 23:06:45 +01:00
Danish Arora
26ab836900
chore: upgrade nwaku to 0.34.0 and update tests suite for compatibility (#2170)
* chore: upgrade nwaku to v0.33.1

* chore: upgrade to nwaku 0.34.0

* feat: connect nwaku nodes amongst each other over relay

* chore(lightpush): use multiple service nodes for lightpush (instead of just one)
- nwaku now expects >=1 nodes at least connected

* chore: all single-node lightpush requests should now be expected to fail

* chore: update sharding tests

* chore: update tests

* chore: improve Docker network config reliability

* chore: deduplicate ecies encrypted payloads

* chore: update to precise expects

* fix: return early if expect passes

* chore: lightpush 5 times instead of 30

* fix: non duplicacy should happen in application-specific scenario

* chore: update mocha config + fix epehermal tests

* chore: reinstall deps after rebase

* chore: attempt stability for test suite

* fix: store tests to now use multiple nodes, delete uneeded test

* fix: memory leak

* chore: switch while loop with timeout-promise

* chore: remove redundant nodes startup

* chore: add delays for nwaku setup
2025-02-17 19:21:33 +05:30
Danish Arora
0a0a92bccb
feat: @waku/rln (#2244)
* chore: setup rln as a new package

* chore: migrate src

* fix: wasm loading, tests, config

* chore: fix Karma CI

* fix: bundler

* chore: copy dist resources

* chore(rln): enable all tests

* chore: increase karma timeouts
2025-02-11 15:28:00 +05:30
Danish Arora
becb46f3a5
chore(sds): use allure reporting (#2255) 2025-02-11 00:56:00 +05:30
Arseniy Klempner
be93e4b71f
feat(sds): migrate bloomfilter to bigint and import hashn function from nim
Uses an array of bigint to store sufficient bits in bloom filter.
Updates all arithmetic to explicitly cast to bigint where necessary.
Makes the hashn function for bloomfilter a parameter.
Adds an implementation of hashn generated using nim compiler.
Adds tests.
2025-02-05 17:27:41 -08:00
Sasha
3136f3a704
feat: add HealthIndicator with simplified logic and testing (#2251)
* implement HealthIndicator

* up libp2p interface version

* up lock

* remove unused tests

* expose HealthIndicator from Waku

* update test, add start and stop

* fix error handling
2025-02-05 13:24:50 +01:00
Sasha
fc93fae873
feat: move Peer to PeerId (#2246)
* feat: move Peer to PeerId

* up tests

* update tests
2025-02-05 11:21:22 +01:00
fbarbu15
2a7f4b6e86
chore: add back allure (#2247) 2025-02-03 13:59:55 +05:30
Sasha
62f93dc842
feat: improve peer manager and re-integrate to light push (#2191)
* up lock

* make ConnectionManager use ctor

* reform connection manager configurations

* remove log param from peerManager

* make PeerManager use only ConnectionManager, move getPeers to ConnectionManager, remove not needed code

* remove allPeers and connectedPeers from BaseProtocolCore, update tests, add getPeers for IWaku

* use only one peerManager from Waku object

* remove IBaseProtocolSDK and merge with PeerManager

* re-implement peerManager, remove ProtocolUseOptions

* remove not needed test, up lock

* update deps and lock

* remove old test for peerManager, fix check and spell

* rename to getConnectedPeers

* feat: improve filter subscriptions (#2193)

* add message cache to Filter

* remove WakuOptions and use only ProtocolCreateOptions

* move subscribe options to createLightNode Fitler protocol options

* rename SubscriptionManager to Subscription

* rename to CreateNodeOptions

* add warning

* feat: introduce subscription manager (#2202)

* feat: inroduce subscription manager

* fix: make pipeline succeed (#2238)

* fix test

* use hardcoded value

* update playwright

* fix test:browser

* up lock

* make peer retrieval probabilistic

* add comments

* up lightpush tests

* add tests for peer_manager, improve folder structure

* create named files for protocols

* create named files, simplify project structure

* remove only
2025-01-31 00:16:00 +01:00
Ivan FB
d706fe146c
replace pubsub topic param with clusterid plus shard (#2239)
We are in the process of deprecating pubsub-topic param from nim-waku.
The pubsub-topic shouldn't be used to start nim-waku and instead, a
combination of 1 cluster-id plus n shard should be given. This aims to
start using the new added param: shard.
2025-01-30 18:16:07 +05:30
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00
Danish Arora
4153396014
feat(store): allow specifying node to use (#2192)
* feat: API allows using specific nodes for protocol

* chore: dial provided now

* chore: store uses the node

* chore: update API

* chore: use protocol-level class for getting peer

* chore: use ConnectionManager for dial ops

* chore: address comments

* chore: fix type error

* chore: Waku.dial() proxies through ConnectionManager.dialPeer

* chore: fix dial

* chore: add ts-doc for dialPeer()

* chore: remove log

* chore: reduce ts-doc for an internal function

* chore: address comments

* chore: return types from connmanager.dialpeer()

* chore: reduce diff by not introducing breaking changes
2025-01-28 17:57:49 +05:30
Danish Arora
f3aed7074b
chore: switch out of allure reporters (#2222)
* chore: switch out of allure

* chore: fix config

* fix: test file patterns
2025-01-27 21:14:34 +05:30
Sasha
f34fc4b244
fix: remove peer deps (#2200) 2025-01-16 14:27:46 +01:00
Sasha
b5d2c013a1
fix access to process variable (#2201) 2025-01-16 13:32:40 +01:00
Sasha
88e33a90fd
fix: remove window reference and improve waitForRemotePeer (#2194)
* fix: remove window reference and improve waitForRemotePeer

* remove only

* up lock

* debug: try * as version

* return version

* up lock
2024-11-08 12:34:31 +07:00
Danish Arora
7c0ce7b2ec
chore: upgrade to libp2p v2 (#2143)
* chore: upgrade libp2p to v2 and related deps

* chore: fix ENR

* chore(core): remove CustomEvent polyfill import

* chore: `peer-id-factory` has been removed

* chore(discovery): fix local-cache & remove CustomEvent imports

* chore(sdk): update config

* chore(tests): update tests without peer-id-factory

* fix: spec tests

* chore: fix test

* chore: upgrade dataset-core

* chore: upgrade libp2p and stale references

* chore: upgrade playwright

* chore: rm console log

* fix: lock
2024-10-21 16:43:24 +05:30
Sasha
c591e6d294
chore: release master (#2135) 2024-10-17 14:06:38 +02:00
Sasha
ded994f8ec
fix: attempt to fix some of the Filter issues (#2183)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* turn off check for missing messages

* fix recurring ping

* add useful logs

* skip tests

* remove comment

* feat: check filter subscriptions against lightPush (#2185)
2024-10-17 01:01:21 +02:00
Sasha
4049123f14
feat!: lighten retry logic for LightPush (#2182)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* skip tests
2024-10-17 00:49:24 +02:00
Danish Arora
19a5d29aa7
chore: change filter ping interval to 1 min (#2184) 2024-10-16 15:30:56 +05:30
Danish Arora
b2efce5ec2
feat(filter): enhancing protocol peer management with mutex locks (#2137)
* chore: improvements

* chore: add logs for subscription maintenance

* chore: update logging

* chore: trimming down BaseProtocolCore

* chore: track peers in a hashmap instead of array

* chore: peer mgmt responds to conenction/disconnection and improve logging

* feat: add mutex locks to tackle race conditions over shared state

* fix: build

* chore: some mutex lock-release improvements

* feat: peer manager

* chore: rm tests for remove internal util

* chore: update HealthManager updates

* chore: update tests

* rm: only

* fix: hasPeers management

* chore: add modularity to getting connected peers

* chore: improve logs & add debug

* chore: renewal doesnt disconnect, only removes

* chore: await for sequential operations

* chore: add TODO

* chore: minor improvements

* chore: fix rebase

* chore: update playright

* chore: remove additional arg

* chore: update interafce

* feat(peer-manager): unit tests

* chore: improve hasPeers()

* chore: update lockfile

* feat: Filter reacts to peer:disconnect event, add tests

* chore: fix lock

* chore: update playright

* chore: update protocol health for lightpush

* chore: remove .only

* chore: address comments and improvements

* fix: tsconfig
2024-10-10 23:47:12 +02:00
Sasha
75fcca4cd9
feat: replace waitForRemotePeers() with waku.waitForPeer() method (#2161)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* add I prefix to Waku interface

* implement waku.connect method

* add docs to IWaku interface

* remove export and usage of waitForRemotePeer

* move wait for remote peer related to Realy out of @waku/sdk

* change tests to use new API

* fix linting

* update size limit

* rename .connect to .waitForPeer

* export waitForRemotePeer and mark as deprecated

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer (#2163)

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer

* add waitForRemote UTs

* remove junk

* feat: expose peerId and protocols from WakuNode (#2166)

* chore: expose peerId and protocols from WakuNode

* remove unused method

* move to private method

* rename to waitForPeers

* up test
2024-10-09 00:43:34 +02:00
Sasha
d37e0245cf
feat: confirm metadata and protocols needed in waitForRemotePeer (#2160)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* imrpove iteration for existing connections

* address protocol adverisement in CI

* add protocols needed

* add missing protocols

* make lightpush and filter default for tests

* up
2024-10-05 01:49:54 +02:00
Sasha
2be0e81a0a
chore: rename IProtocolSDK interfaces to IProtocol naming convention (#2159)
* chore: rename IProtocolSDK interfaces to IProtocol
2024-10-04 13:53:54 +02:00
Sasha
1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Sasha
b93134a517
fix: peer renewal connection drop & stream management (#2145)
* fix: peer renewal connection drop

* fix stream manager

* fix over iteration during stream creation

* remove timeout and use only open peers

* add logs

* refactor code, add tests

* debug test

* up debug

* remove debug, supress check for timestamps

* remove only

* add more debug

* remove debug

* remove check for timestamps
2024-10-01 12:54:55 +02:00