927 Commits

Author SHA1 Message Date
Danish Arora
ff8b4e5fdb
fix: MemLocalHistory uses optional chaining 2025-12-10 15:20:28 -05:00
Danish Arora
4757322489
chore: update interface to type as we're not implementing it 2025-12-10 15:20:27 -05:00
Danish Arora
a3202b58dd
chore: interface abstractions 2025-12-10 15:20:27 -05:00
Danish Arora
e54dacddd7
chore: remove unused exports 2025-12-10 15:20:27 -05:00
Danish Arora
503c348b79
feat: Introduce PersistentStorage for message history management and refactor MemLocalHistory to support optional persistent storage. 2025-12-10 15:20:27 -05:00
Danish Arora
ad4973aa20
test: local storage specific test 2025-12-10 15:20:27 -05:00
Danish Arora
d6412e5536
test: add unit tests for localStorage persistence and error handling in message channels 2025-12-10 15:20:26 -05:00
Danish Arora
d52ea9fd2c
tests: use memLocalHistory for message_channel.spec.ts 2025-12-10 15:20:26 -05:00
Danish Arora
581430ab43
feat: Conditionally set default history storage to localStorage for improved compatibility. 2025-12-10 15:20:26 -05:00
Danish Arora
15f1a22f2e
feat: remove storageKeyPrefix and getDefaultHistoryStorage, simplifying history storage initialization to default to localStorage 2025-12-10 15:20:26 -05:00
Danish Arora
322240b2d0
refactor: Remove silent error suppression for persistent history storage operations. 2025-12-10 15:20:26 -05:00
Danish Arora
b8ee363cb1
refactor: rename persistence methods from persist/restore to save/load 2025-12-10 15:20:25 -05:00
Danish Arora
032c3f5faf
refactor: rename storageKey to storageKeyPrefix and update storage key generation. 2025-12-10 15:20:25 -05:00
Danish Arora
ca24c09688
feat(history): introduce ILocalHistory interface and refactor PersistentHistory to use composition over inheritance for history management. 2025-12-10 15:20:25 -05:00
Danish Arora
454dc4a93b
refactor: simplify localHistory initialization by removing intermediate constant 2025-12-10 15:20:25 -05:00
Danish Arora
dad9cffdbe
feat: persistent history for SDS 2025-12-10 15:20:24 -05:00
Arseniy Klempner
f2ad23ad43
feat(rln)!: generate contract types, migrate from ethers to viem (#2705)
* feat: use wagmi to generate contract types

* feat: migrate rln from ethers to viem

* fix: remove .gitmodules

* fix: update readme

* fix: refactor to use a single viem client object

* fix: update comments, tsconfig

* feat: remove membership event tracking

* fix: script name in package.json and readme

* fix: only allow linea sepolia

* fix: consolidate viem types, typed window

* fix: use viem to infer type of decoded event

* fix: use js for generate abi script

* feat: generate abi and build rln package as release condition

* fix: check that eth_requestAccounts returns an array

* fix: handle error messages

* fix: use https instead of git for cloning in script

* fix: add warning annotations for contract typings check

* fix: install deps for rln package before building

* fix: use pnpm when installing rln contracts

* fix: use workspace flag to run abi script

* fix: add ref to checkout action

* fix: include pnpm in ci
2025-12-01 17:32:35 -08:00
Hanno Cornelius
788f7e62c5
feat: incorporate sds-r into reliable channels (#2701)
* wip

* feat: integrate sds-r with message channels

* fix: fix implementation guide, remove unrelated claude file

* feat: integrate sds-r within reliable channels SDK

* fix: fix import, export

* fix: fix build errors, simplify parallel operation

* fix: sigh. this file has 9 lives

* fix: simplify more

* fix: disable repair if not part of retrieval strategy

* fix: remove dead code, simplify

* fix: improve repair loop

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* chore: make retrievalStrategy mandatory argument

* chore: add repair multiplier, safer checks

---------

Co-authored-by: fryorcraken <commits@fryorcraken.xyz>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2025-11-21 15:03:48 +00:00
fryorcraken
e5f51d7df1
feat: Reliable Channel: Status Sync, overflow protection, stop TODOs (#2729)
* feat(sds): messages with lost deps are delivered

This is to re-enable participation in the SDS protocol. Meaning the
received message with missing dependencies becomes part of the causal
history, re-enabling acknowledgements.

* fix(sds): avoid overflow in message history storage

* feat(reliable-channel): Emit a "Synced" Status with message counts

Return a "synced" or "syncing" status on `ReliableChannel.status` that
let the developer know whether messages are missing, and if so, how many.

* fix: clean up subscriptions, intervals and timeouts when stopping

# Conflicts:
#	packages/sdk/src/reliable_channel/reliable_channel.ts

* chore: extract random timeout

* fix rebase

* revert listener changes

* typo

* Ensuring no inconsistency on missing message

* test: streamline, stop channels

* clear sync status sets when stopping channel

* prevent sync status event spam

* test: improve naming

* try/catch for callback

* encapsulate/simplify reliable channel API

* sanity checks

* test: ensure sync status cleanup
2025-11-16 08:57:12 +11:00
fryorcraken
84a6ea69cf
fix: cleanup routines on reliable channel and core protocols (#2733)
* fix: add stop methods to protocols to prevent event listener leaks

* fix: add abort signal support for graceful store query cancellation

* fix: call protocol stop methods in WakuNode.stop()

* fix: improve QueryOnConnect cleanup and abort signal handling

* fix: improve MissingMessageRetriever cleanup with abort signal

* fix: add stopAllRetries method to RetryManager for proper cleanup

* fix: implement comprehensive ReliableChannel stop() with proper cleanup

* fix: add active query tracking to QueryOnConnect and await its stop()

* fix: add stop() to IRelayAPI and IStore interfaces, implement in SDK wrappers

* align with usual naming (isStarted)

* remove unnecessary `await`

* test: `stop()` is now async

* chore: use more concise syntax

---------

Co-authored-by: Levente Kiss <levente.kiss@solarpunk.buzz>
2025-11-13 12:32:15 +11:00
Sasha
101ffe8a04
chore: release master (#2663) 2025-11-04 15:56:42 +01:00
Hanno Cornelius
5334a7fcc9
feat: add SDS-Repair (SDS-R) to the SDS implementation (#2698)
* wip

* feat: integrate sds-r with message channels

* feat: add SDS-R events

* fix: fixed buffer handling incoming and outgoing

* fix: more buffer fixes

* fix: remove some magic numbers

* fix: buffer optimisation, backwards compatible senderId

* fix: fix implementation guide, remove unrelated claude file

* fix: further buffer optimisations

* fix: linting errors

* fix: suggestions from code review

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* fix: remove implementation guide

* fix: build errors, remove override, improve buffer

* fix: consistent use of MessageId and ParticipantId

* fix: switch to conditionally constructed from conditionally executed

---------

Co-authored-by: fryorcraken <commits@fryorcraken.xyz>
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2025-10-28 10:27:06 +00:00
Arseniy Klempner
115cdd28fe
chore: update hardcoded version of nwaku to 0.36.0, remove unused ci job (#2710)
* fix: update hardcoded version of nwaku to 0.36.0

* fix: remove unused/outdated rln-sync-tree job
2025-10-27 17:02:48 -07:00
Arseniy Klempner
0df18b2a75
feat: create @waku/run package for local dev env (#2678)
* feat: create @waku/run package for local dev env

* chore: add @waku/run to release please config

* feat: test @waku/run with playwright

* fix: don't run waku/run tests in CI

* fix: cache images so docker-compose can work offline

* feat: set nodekey and staticnode flags for each nwaku node

* fix: use constants for node ids

* chore: set directories for running via npx

* fix: remove .env, support env vars for nwaku ports

* fix: use separate db (same instance) for each node

* feat: add command to test dev env

* chore: use package version in container name

* fix: replace hardcoded WS/REST ports with constants/env vars

* chore: clean up README

* fix: refactor config printing into own function

* fix: add run package to release please manifest

* fix: defer to root folder gitignore/cspell

* fix: update node version and remove tsx

* fix: remove browser tests and express dep

* fix: replace magic values with constants

* fix: move to root .gitignore

* fix: move cspell to root
2025-10-22 21:38:28 -07:00
fryorcraken
b8a9d132c1
chore: npm publication (#2688)
* chore: npm publication

Fixing npm publication and warnings

* Upgrade workflow to use trusted publishing

https://docs.npmjs.com/trusted-publishers

* bump node js to 24

To avoid having to reinstall npm in pre-release for npmjs trusted publishers
2025-10-21 16:35:01 +11:00
Sasha
37c6c1e529
chore: expose sdk from waku/react (#2676) 2025-10-09 00:38:54 +02:00
Sasha
ad0bed69ba
feat: add waku/react package and make it compatible with React frameworks (#2656)
* chore: add waku/react package

* fix check

* remove not needed logic from waku/react package

* make it compatible with expo/next

* add to release please

* remove tests
2025-10-08 15:37:49 +02:00
Arseniy Klempner
d803565b30
feat(browser-tests): simplify, refactor, update dockerized browser node (#2623)
* feat(browser-tests): simplify, refactor, update dockerized browser node

* Update packages/browser-tests/web/index.ts

* fix: remove comments and console.logs from tests

* fix: add temporary logging

* fix: debugging static sharding

* fix: replace console with logger

* fix: remove use of any

* fix: log dial error

* fix: replace any with libp2p options

* fix: remove unused logic around sourcing address.env

* fix: uncomment log

* fix: add more logging and fix tests

* feat: add types for test-config

* fix: add types to server.ts

* fix: remove more uses of any

* fix: remove use of any in endpoint handlers
2025-10-07 10:54:19 -07:00
fryorcraken
e92f6a2409
feat!: do not send sync messages with empty history (#2658)
* feat!: do not send sync messages with empty history

A sync message without any history as no value. If there are no messages in the channel, then a sync messages does not help.

If there are messages in the channel, but this participant is not aware of them, then it can confuse other participants to assume that the channel is empty.

* fix test by adding a message to channel history

* make `pushOutgoingSyncMessage` return true even if no callback passed
2025-10-02 15:17:10 +10:00
fryorcraken
c0ecb6abba
fix!: SDS lamport timestamp overflow and keep it to current time (#2664)
* fix!: avoid SDS lamport timestamp overflow

The SDS timestamp is initialized to the current time in milliseconds, which is a 13 digits value (e.g. 1,759,223,090,052).

The maximum value for int32 is 2,147,483,647 (10 digits), which is clearly less than the timestamp.
Maximum value for uint32 is 4,294,967,295 (10 digits), which does not help with ms timestamp.

uint64 is BigInt in JavaScript, so best to be avoided unless strictly necessary as it creates complexity.
max uint64 is 18,446,744,073,709,551,615 (20 digits).

Using seconds instead of milliseconds would enable usage of uint32 valid until the year 2106.

The lamport timestamp is only initialized to current time for a new channel. The only scenario is when a user comes in a channel, and thinks it's new (did not get previous messages), and then starts sending messages. Meaning that there may be an initial timestamp conflict until the logs are consolidated, which is already handled by the protocol.

* change lamportTimestamp to uint64 in protobuf

* lamport timestamp remains close to current time
2025-10-02 09:07:10 +10:00
fryorcraken
593bc45225
feat: reliable channels search up to 30 days to find message (#2657)
* feat: query on connect stops on predicate

* test: query on connect stops at predicate

* feat: reliable channels search up to 30 days to find message

Queries stop once a valid sync or content message is found in the channel.

* fix: protect against decoding exceptions

* stop range queries on messages with a causal history
2025-10-01 21:35:52 +10:00
Arseniy Klempner
bbcfc94879
feat(rln)!: use zerokit for credential generation (#2632)
Co-authored-by: Danish Arora <danisharora099@gmail.com>
2025-09-30 16:49:18 -07:00
Sasha
016a25d578
chore: stop updating TTL for peer store (#2653) 2025-10-01 00:33:26 +02:00
Sasha
79dd001b1f
chore: release master (#2576) 2025-09-20 11:37:22 +02:00
Sasha
11d84ad342
feat: implement peer-store re-bootstrapping (#2641)
* implement peer-store re-bootstrapping

* add peer cache support

* implement TTL update for open connections, add re-bootstrapping in case reaches zero peers

* fix query tests, skip missing message retrival

* up tests

* up sds tests

* skip

* skip
2025-09-20 09:40:51 +02:00
fryorcraken
cb3af8cd4d
fix(sds): initialize lamport timestamp with current time (#2610) 2025-09-11 18:06:54 +10:00
fryorcraken
4d5c152f5b
feat: introduce reliable channels (#2526)
* SDS: pushOutgoingMessage is actually sync

* SDS: ensure that `ContentMessage` class is stored in local history with `valueOf` method

* feat: introduce reliable channels

Easy to use Scalable Data Sync (SDS, e2e reliability) wrapper, that includes:
- store queries upon connection to store nodes
- store queries to retrieve missing messages

* remove `channel` prefix

* attempt to improve performance when processing a lot of incoming messages

* test: split test file

* use index.ts for re-export only.

* improve if condition

* use getter for isStarted

* waku node already auto-start

* rename send

* fix lightPush.send type post rebase

* test: remove extra console.log

* SDS: emit messages as missing as soon as they are received

* make configurable elapse time for task process

* typo

* use string instead of enum for event types

* ReliableChannel.send returns the message id
2025-09-09 12:43:48 +10:00
Sasha
8f09f5fa5a
fix: nwaku master interop tests (#2625)
* fix REST endpoint

* disable px in peer cache test
2025-09-08 22:48:18 +02:00
Arseniy Klempner
16253026c6
feat: implement lp-v3 error codes with backwards compatibility (#2501)
* feat: implement LightPush v3 protocol support

Add comprehensive LightPush v3 protocol implementation with:

Core Features:
- LightPush v3 protocol codec and multicodec detection
- Status code-based error handling and validation
- Protocol version inference and compatibility layers
- Enhanced error types with detailed failure information

Protocol Support:
- Automatic v3/v2 protocol negotiation and fallback
- Status code mapping to LightPush error types
- Protocol version tracking in SDK results
- Mixed protocol environment support

Testing Infrastructure:
- Comprehensive v3 error code handling tests
- Mock functions for v3/v2 response scenarios
- Protocol version detection and validation tests
- Backward compatibility verification

Implementation Details:
- Clean separation between v2 and v3 response handling
- Type-safe status code validation with isSuccess helper
- Enhanced failure reporting with protocol version context
- Proper error propagation through SDK layers

This implementation maintains full backward compatibility with v2
while providing enhanced functionality for v3 protocol features.

* feat: handle both light push protocols

* fix: unsubscribe test

* feat: consolidate lpv2/v3 types

* feat(tests): bump nwaku to 0.36.0

* fix: remove extraneous exports

* fix: add delay to tests

* fix: remove protocol result types

* feat: consolidate light push codec branching

* fix: revert nwaku image

* fix: remove multicodec

* fix: remove protocolversion

* feat: simplify v2/v3 branching logic to use two stream managers

* fix: remove unused utils

* fix: remove comments

* fix: revert store test

* fix: cleanup lightpush sdk

* fix: remove unused util

* fix: remove unused exports

* fix: rename file from public to protocol_handler

* fix: use proper type for sdk result

* fix: update return types in filter

* fix: rebase against latest master

* fix: use both lightpush codecs when waiting for peer

* fix: handle both lp codecs

* fix: remove unused code

* feat: use array for multicodec fields

* fix: add timestamp if missing in v3 rpc

* fix: resolve on either lp codec when waiting for peer

* fix: remove unused util

* fix: remove unnecessary abstraction

* feat: accept nwaku docker image as arg, test lp backwards compat

* fix: revert filter error

* feat: add legacy flag to enable lightpushv2 only

* Revert "feat: accept nwaku docker image as arg, test lp backwards compat"

This reverts commit 857e12cbc73305e5c51abd057665bd34708b2737.

* fix: remove unused test

* feat: improve lp3 (#2597)

* improve light push core

* move back to singualar multicodec property, enable array prop only for light push

* implement v2/v3 interop e2e test, re-add useLegacy flag, ensure e2e runs for v2 and v3

* fix v2 v3 condition

* generate message package earlier

* add log, fix condition

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: Sasha <oleksandr@status.im>
2025-09-05 00:52:37 +02:00
Sasha
a7f30b1211
feat: add debounce to health indicator (#2594) 2025-09-04 00:49:10 +02:00
Sasha
2fba052b8b
feat: add start/stop to filter (#2592) 2025-08-30 01:32:59 +02:00
fryorcraken
914beb6531
fix: (sds) ensure incoming messages have their retrieval hint stored (#2604)
* SDS: export `MessageId`

* SDS: attach retrieval hints to incoming messages

* sds: ensure items are ordered by timestamp

* test: sds: avoid using "as any" as it bypasses type checks

* test: filter: avoid using "as any" as it bypasses type checks

* test: fix tests without introducing proxy
2025-08-28 15:57:23 +10:00
fryorcraken
8542d04bf5
feat: query on connect (#2602)
* feat: query on connect

Perform store time-range queries upon connecting to a store node.
Some heuristics are applied to ensure the store queries are not too frequent.

* make `maybeQuery` private

* query-on-connect: use index.ts only for re-export

* query-on-connect: update doc
2025-08-28 13:58:16 +10:00
fryorcraken
826aedd558
log: logs are already prefixed with waku (#2603) 2025-08-27 23:32:41 +10:00
fryorcraken
0dfbcf6b6b
feat: StoreConnect events (#2601)
* store connect evt: use enum instead of free strings for Waku event types

* store connect evt: more accurate enum name

* store connect evt: add store connect event on peer manager

* store connect evt: simplify logic statements

* store connect evt: test store connect

* store connect evt: export event types

* test: use enum

* Shorter name for waku events
2025-08-27 12:29:22 +10:00
fryorcraken
78c856d079
fix: remove sharding circular dependency (#2590) 2025-08-25 10:49:05 +10:00
Sasha
836d6b8793
feat: expose message hash from IDecodedMessage (#2578)
* expose message hash from IDecodedMessage

* up mock

* optimize hashing and add tests
2025-08-21 11:58:43 +02:00
Sasha
c8dfdb1ace
fix: make health events emission consistent (#2570)
* fix health events emission

* fix bloom test flakiness due to bloom filter properties
2025-08-20 13:13:38 +02:00
Sasha
26de2d11c8
chore: release master (#2515) 2025-08-15 21:56:59 +02:00
Sasha
dfb2baf004
fix: prevent setting shard info from PX if it exists (#2561)
* prevent setting px shard info

* address review
2025-08-15 00:30:07 +02:00