838 Commits

Author SHA1 Message Date
fryorcraken
6d55af947e
fix!: remove node level pubsub topic concept
For an edge node, there is no such thing as a "pubsub topic configuration". An edge node should be able to operate for any possible shard, and it is a per-protocol matter (eg send message with light push).

A relay node do subscribe to shards, but in this case, even metadata protocol does not need to advertise them, this is already handled by gossipsub.

Only service node should advertise their shards via metadata protocol, which is out of scope for js-waku.

# Conflicts:
#	packages/interfaces/src/connection_manager.ts
2025-07-17 09:57:15 +10:00
fryorcraken
188e4bf928
fix typo 2025-07-17 09:56:15 +10:00
Sasha
ed389ccbc9
feat: add recovery and connection maintenance (#2496)
* add FF for auto recovery

* implement connection locking, connection maintenance, auto recovery, bootstrap connections maintenance and fix bootstrap peers dropping

* add ut for peer manager changes

* implement UT for Connection Limiter

* increase connection maintenance interval

* update e2e test
2025-07-17 01:15:36 +02:00
Danish Arora
7f7f772d93
feat(rln): price calculator for rate limits (#2480)
* chore: add ABI for PriceCalculator

* chore: rename LINEA_CONTRACT to RLN_CONTRACT

* chore: add price calculator & test

* fix: import

* chore: convert e2e test to unit

* fix: test
2025-07-16 15:16:13 +05:30
Sasha
35acdf8fa5
feat: don't dial peers that failed before, make dialer use dial queue (#2478)
* make dialer use dial queue

* skip undiable peers

* cover new cases in tests

* expose dialer config at connection manager

* update tests with new config

* add more tests
2025-07-15 23:13:58 +02:00
Sasha
27292edabc
feat!: unify events under one source (#2473)
* move health indicator under waku.events and expose from Waku as a value

* update tests

* make new type for libp2p event handlers

* fix types
2025-07-15 00:59:45 +02:00
Arseniy Klempner
94788d0dfc
feat(proto): add light push v3 definition 2025-07-14 08:39:41 -07:00
Danish Arora
9b0c5e8311
fix: idCommitmentBigInt must always be less than the contract Q (#2394)
* chore: idCommitmentBigInt validates against contract Q

* chore: fix linting

* chore: add log

* chore: rename Q and make sync

* fix: test

* chore: remove stubbed contract test

* chore: hardcode default constant for Q

* use non deprecated sha256

* chore: use full 32 bytes for bigint

* chore: all storage in LE, but smart contract interactions in BE

* chore: remove references to idCOmmitmentBigInt in Identity

* chore: don't fetch Q from contract

* chore: ByteUtils as a class

* chore: store Identity in BE, convert during Keystore

* chore: add IDCommitmentBigInt part of Identity

* chore: minor improvements

* chore: switch idTrapdoor to LE

* chore: add logs

* chore: rename `DEFAULT_Q` to `RLN_Q`

* chore: rm spec test

* chore: improve modulo logging

* fix(tests): add IDCommitmentBigInt
2025-07-14 16:35:34 +05:30
fbarbu15
5d8cfff7eb
chore: add throughput reliability tests (#2444)
* chore: add throughput reliability tests

* chore: add capability to run all
2025-07-14 11:38:42 +03:00
Sasha
f75634d9c5
feat: update logger format (#2472) 2025-07-11 22:15:22 +02:00
Sasha
c7682ea67c
feat!: re-architect connection manager (#2445)
* remove public pubsub field and redundant util

* add hangUp and improve dial operations, improve keepAliveManager and remove unused method, move utils and add tests

* improve public dial method to start keep alive checks

* move dial method

* implement discovery dialer

* implement discovery dialer with queue with tests

* add discovery dialer e2e tests, change local discovery log tag, update other tests

* remove comment

* add issue link, remove only

* implement shard reader component

* create evetns module, remove unused connection manager events and related tests

* implement network indicator

* implement connection limiter, change public API of connection manager, implement recovery strategy

* decouple keep alive maanger

* add connection manager js-doc

* refactor keep alive manager, cover with tests

* add tests for connection manager main facade

* add tests for connection limiter

* add e2e tests for connection manager modules
pass js-waku config during test node init
remove dns discovery for js-waku

* restructure dialing tests

* address last e2e tests

* address review

* add logging for main methods

* decouple pure dialer class, update network monitor with specific metrics

* remove console.log

* remove usage of protocols

* update sdk package tests

* add connect await promise

* add debug for e2e tests

* enable only packages tests

* use only one file

* revert debugging

* up interface for netwrok manager

* add logs

* add more logs

* add more logs

* add another logs

* remove .only

* remove log statements

* skip the test with follow up
2025-07-09 21:23:14 +02:00
Sasha
bfda249aa6
chore: remove fitler codec initiator in filter core (#2443) 2025-07-03 13:28:32 +02:00
Sasha
058f2ff620
feat: make peer manager aware of codec and shard, fix retry manager and shut down subscriptions (#2425)
* implement new peer manager, use in lightPush, improve retry manager and fix retry bug

* fix unsubscribe issue

* remove not needed usage of pubsub, use peer manager in store sdk

* chore: remove deprecated filter implementation

* update tests

* update next filter for new peer manager

* skip IReceiver test, remove unused utility

* remove comment

* fix typo

* remove old connection based peer manager

* update types, export, and edge case for light push

* add retry manager tests

* add new peer manager tests

* refactor tests

* use peer manager events in filter and check for pubsub topic as well

* update test names

* address comments

* unskip Filter e2e test

* address more comments, remove duplication

* skip CI test

* update after merge

* move to peer:idenfity and peer:disconnect events, improve mapping in filter subscriptions

* update tests

* add logs and change peer manager time lock to 10s
2025-07-02 23:03:47 +02:00
Sasha
981248eedd
chore!: remove deprecated filter implementation (#2433)
* chore: remove deprecated filter implementation

* update tests

* skip IReceiver test, remove unused utility

* fix typo

* address comments

* unskip Filter e2e test

* address more comments, remove duplication

* skip CI test
2025-07-02 12:37:54 +02:00
Sasha
cd1d909de3
chore: release master (#2390) 2025-06-23 22:29:40 +02:00
Sasha
f55db3eb4b
feat: shard retrieval for store and store peers selection (#2417)
* feat: implement shard retrieval for store and improve set store peers usage

* remove log

* remove only, improve condition

* implement smarter way to retrieve peers

* up tests

* update mock

* address nits, add target to eslint, revert to es2022
2025-06-23 10:01:54 +02:00
Sasha
fcc6496fef
chore!: upgrade libp2p, nodejs and typescript (#2401)
* remove react native package

* upgrade nodejs

* upgrade libp2p

* upgrade typescript, use 22 node

* up lock

* fix node compatibility

* upgrade playwright

* upgrade proto

* update tests
2025-06-22 17:41:23 +02:00
Danish Arora
fecc026dd7
fix: DIRVER -> DRIVER typo in nwaku (#2406)
* fix: DIRVER -> DRIVER nwaku

* chore: empty commit

* chore: access CI/test env vars and conditionally check test

* fix: undefined nwaku version

* chore: improve splitting

* simplify, test error string only for new versions

---------

Co-authored-by: Sasha <oleksandr@status.im>
2025-06-20 15:34:51 +02:00
Sasha
7c8d1073b0
chore!: remove IBaseProtocol and improve interface on PeerExchange (#2422)
* remove IBaseProtocol

* fix references, interfaces and integration

* fix ci

* up mock

* up lock

* add mock for local storage

* add missing prop, fix tests

* up lock
2025-06-20 12:53:42 +02:00
Arseniy Klempner
8444bc940f
fix: reorder methods by visibility 2025-06-13 15:22:52 -07:00
Arseniy Klempner
a2c3b2e6aa
feat(sds)_: add command queue architecture and improve message handling
- Introduce command queue system for sequential task processing
- Add comprehensive event system for message lifecycle tracking
- Restructure codebase with separate bloom_filter directory
- Export encode/decode helpers for SDS proto messages
- Use Set for deduplication in missing message detection
- Fix sync message handling for empty content messages
- Always emit MissedMessages event even with empty array
- Improve duplicate message detection logic
2025-06-10 20:19:22 -07:00
fbarbu15
4997440225
chore: add reliability tests package with longevity tests (#2361)
* chore: add reliability tests package with longevity tests

* chore: add reliability tests package with longevity tests

* chore: fix ci errors

* chore: fix

* chore: fix timeout
2025-06-05 11:19:00 +03:00
Arseniy Klempner
f649f59e64
fix: rebase package lock 2025-06-03 11:24:37 -07:00
Arseniy Klempner
5f63cb5bfb
fix: remove unnecessary comments from store validation
- Remove inline comments that explain obvious code behavior
- Keep only the reference URL comment which provides useful context
- Address review feedback about excessive commenting
2025-06-03 09:05:54 -07:00
Arseniy Klempner
2d92191029
fix: address review feedback for message hash tests
- Remove all console.log statements (8 instances)
- Update type usage from DecodedMessage[] to IDecodedMessage[]
- Replace assert.equal() with expect().to.equal() for consistency
- Import expect from chai alongside existing imports

These changes address all feedback from PR review without altering test logic.
2025-06-03 09:05:54 -07:00
Arseniy Klempner
9f7a15dfb1
fix(store): update store query validation logic to support msg hash queries 2025-06-03 09:05:54 -07:00
Sasha
1905558753
feat!: re-work messaging parts and sharding (#2399)
* remove message-hash package, move it to core

* add js-doc to message hash

* up

* address type changes

* fix lint
2025-06-03 11:08:02 +02:00
Danish Arora
16328a3f11
feat: enable/disable discoveries without defaultBootstrap: true (#2363)
* feat: enable/disable discoveries without defaultBootstrap: true

* chore: rm .only

* chore: don't redefine default discoveries object

* chore: don't export default const from library

* chore: rename discoveriesEnabled to discovery

* fix: rename

* address comments

* chore: reset default

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>

* chore: address comments

* chore: update tests

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2025-05-30 16:27:00 +05:30
Arseniy Klempner
5ed35471ca
feat: headless app and api for testing js-waku in browser 2025-05-29 10:19:56 -07:00
Danish Arora
3b23bceb9d
feat: refine work with membership info and other meta information (#2341)
* chore: update definitions to match the ABI

* chore: improve exports and types

* chore: use big endian instead of little endian

* chore: improve logging

* chore: update import

* fix: use overloaded function call

* chore: estimate gas

* test

* npm i

* chore: regen locl

* chore: update

* chore: update playwright docker images

* chore: cannot erase membership if it isn't expired/in grace period

* chore: fix condition

* chore: add `getMembershipStatus` method

* refactor(rln): cache min/max rate limits at contract instantiation

* chore: fix accessor

* chore: rename arg

* chore: allow chromebin path through env
2025-05-29 16:09:08 +05:30
Sasha
a4dfd3455c
feat: event based approach to Filter (#2300)
* create new filter api

* implement await on main methods on new Filter

* add info logs in new filter

* add logs to subscription impl

* remove lint supress

* add unit tests

* introduce E2E tests

* update e2e tests and add case for testing filter recovery after nwaku nodes replacement

* add new test cases for max limits and enable decoders as array on new filter

* fix edge case testing, correct test cases

* skip test

* update error message

* up text

* up text

* fix lint

* implement unsubscribeAll

* add js-dock to new filter

* add cspell

* implement TTL set for message history
2025-05-28 00:44:44 +02:00
Sasha
e22f211d46 chore: release master 2025-04-23 07:38:08 +02:00
Arseniy Klempner
408be95a13
feat(sds): add retrieval hint to causal history 2025-04-22 07:45:05 -07:00
Arseniy Klempner
6b4848c853
feat(sds): adds ephemeral messages, delivered message callback and event 2025-04-22 07:16:14 -07:00
Sasha
fd2f5b7b8d
chore: up readme (#2367) 2025-04-18 20:44:22 +02:00
Sasha
551238006c
fix: health indicator properly handle peer:identify event (#2369) 2025-04-18 20:11:09 +02:00
Sasha
3038c48917
feat: introduce createDecoder and createEncoder on IWaku (#2352)
* feat: introduce createDecoder and createEncoder on IWaku

* add tests, refactor

* fix type
2025-04-14 10:46:47 +02:00
Sasha
163bea56c3
chore: skip flaky test due to nwaku bug (#2349)
* chore: skip flaky test due to nwaku bug

* return full test cases for lightpush
2025-04-11 18:30:28 +02:00
Sasha
3793e6f5c0
chore: update interop test suit for latest nwaku (0.35.1) (#2345)
* update Filter test suit, make service nodes connected to each other, remove single node Filter test suit, use 0.35 nwaku image

* update light push tests

* improve auto shard tests

* update static sharding test

* skip blocked tests

* fix test

* remove usage of pusubtopic with nwaku

* remove comment
2025-04-11 01:34:11 +02:00
Danish Arora
28f28d0d36
fix(rln): update types to match nwaku (#2339)
* chore: update types to match nwaku

* chore: add userMessageLimit // not in spec
2025-04-10 14:42:57 +05:30
Danish Arora
4adf8706c3
feat(rln): create CredentialsManager without Zerokit (#2295)
* chore: update ABI

* chore: update contract address and chain ID to Linea Sepolia

* chore: improve error handling

* fix: bigint conversion

* chore: update tests

* chore: clean comments

* chore: export keystore types

* chore: update README with contract address

* tests: add reusable mock functions

* chore: LINEA_CONTRACT instead of SEPOLIA_CONTRACT

* chore: add linea to cspell

* chore: add rateLimit to MembershipInfo

* fix: determine start options

* feat: RLNLight

* chore: fix

* chore: export

* fix: returns for ratelimit

* chore: big number conversions

* chore: rename RLNLight to CredentialsManager, add logs

* chore: setup and use rln_base_contract

* chore: use CredentialsManager for rln.ts

* chore: public methods written above private methods

* fix: rate limit getter

* chore: simplify getters/setters

* chore: insert empty line
2025-04-07 16:04:06 +05:30
Danish Arora
a8ff776962
chore(rln): update ABI and contract address to Linea Sepolia, enhancements (#2294)
* chore: update ABI

* chore: update contract address and chain ID to Linea Sepolia

* chore: improve error handling

* fix: bigint conversion

* chore: update tests

* chore: clean comments

* chore: export keystore types

* chore: update README with contract address

* tests: add reusable mock functions

* chore: LINEA_CONTRACT instead of SEPOLIA_CONTRACT

* chore: add linea to cspell

* chore: add rateLimit to MembershipInfo

* fix(tests): rate limit additions

* chore: update gitignore
2025-03-31 14:32:29 +05:30
Sasha
ea6daae927
chore: release master (#2311) 2025-03-24 14:41:33 +01:00
Sasha
1ee77c0423
fix: downgraid chainsafe (#2316)
* fix: downgraid chainsafe

* up lock

* up lock

* bump playwright

* add override section
2025-03-24 13:07:29 +01:00
fryorcraken
d27db21ba5
fix: do not use waku test fleet as default bootstrap (#2312)
The waku test fleet is unstable and should not be used as boostrap.

The issue is that DNS Discovery looks for "2" nodes and stop there. Meaning that if those "discovered" nodes are not reachable (ie, coming from test suite). js-waku does not connect.
2025-03-20 16:32:27 +01:00
Sasha
e0b8bd8c9c
chore: update release please and add packages (#2305) 2025-03-20 00:40:34 +01:00
fryorcraken
edfb56243d
doc: build first to avoid not found deps issues (#2307)
* doc: build first to avoid not found deps issues

* fix: Remove XMLHttpRequest usage

XMLHttpRequest API is deprecated and not available in browser extensions.

Replace the culprit dependency with a more modern one.

Some options are removed. The assumption is that nobody uses them. It can always be added if a developer wants the flexibility.

* test: simplify test

The test focus on testing DNS Discovery, there is no need to also have a nwaku local node for bootstrap.

Bootstrap on nwaku local node is used in many other tests.
2025-03-19 09:43:41 +01:00
Arseniy Klempner
13ae5d4f73
feat(sds): send and receive sync messages 2025-03-13 11:29:55 -07:00
Arseniy Klempner
f7666a6588
feat(sds): adds logic to sweep incoming and outgoing buffers 2025-03-03 22:09:44 -08:00
Danish Arora
6fc6bf3916
feat(rln): migrate from v1 to v2, rate limiting, memberships, test coverage (#2262)
* chore: update ABIs and deployed address

* chore: remove storage contract references

* feat: upgrading adapter to basic rlnv2

* feat: rate limit

* chore: upgrade packages revert

* fix: tests

* chore: remove uneeded file

* feat(rln): implement RLNv2 rate limiting and membership states

- Add rate limit validation and handling in proof generation/verification
- Implement membership lifecycle state management (Active/GracePeriod/Expired)
- Add new membership management methods:
  - getMembershipInfo
  - extendMembership
  - eraseMembership
  - registerMembership
- Update proof verification to include rate limit checks
- Refactor message serialization to include rate limit data

Breaking changes:
- verifyWithRoots now takes roots as array instead of spread parameters
- Proof verification methods now accept optional rateLimit parameter

* fix: typo

* chore: add to cspell

* chore: reduce diff

* chore: simplify subdir for abi

* chore: address comments

* chore: simplify access to variables

* chore: address comments

* chore: simplify constants

* chore: add error handling

* chore: change rln v2 references to rln

* fix: check
2025-03-03 18:14:06 +05:30