22 Commits

Author SHA1 Message Date
Arseniy Klempner
16253026c6
feat: implement lp-v3 error codes with backwards compatibility (#2501)
* feat: implement LightPush v3 protocol support

Add comprehensive LightPush v3 protocol implementation with:

Core Features:
- LightPush v3 protocol codec and multicodec detection
- Status code-based error handling and validation
- Protocol version inference and compatibility layers
- Enhanced error types with detailed failure information

Protocol Support:
- Automatic v3/v2 protocol negotiation and fallback
- Status code mapping to LightPush error types
- Protocol version tracking in SDK results
- Mixed protocol environment support

Testing Infrastructure:
- Comprehensive v3 error code handling tests
- Mock functions for v3/v2 response scenarios
- Protocol version detection and validation tests
- Backward compatibility verification

Implementation Details:
- Clean separation between v2 and v3 response handling
- Type-safe status code validation with isSuccess helper
- Enhanced failure reporting with protocol version context
- Proper error propagation through SDK layers

This implementation maintains full backward compatibility with v2
while providing enhanced functionality for v3 protocol features.

* feat: handle both light push protocols

* fix: unsubscribe test

* feat: consolidate lpv2/v3 types

* feat(tests): bump nwaku to 0.36.0

* fix: remove extraneous exports

* fix: add delay to tests

* fix: remove protocol result types

* feat: consolidate light push codec branching

* fix: revert nwaku image

* fix: remove multicodec

* fix: remove protocolversion

* feat: simplify v2/v3 branching logic to use two stream managers

* fix: remove unused utils

* fix: remove comments

* fix: revert store test

* fix: cleanup lightpush sdk

* fix: remove unused util

* fix: remove unused exports

* fix: rename file from public to protocol_handler

* fix: use proper type for sdk result

* fix: update return types in filter

* fix: rebase against latest master

* fix: use both lightpush codecs when waiting for peer

* fix: handle both lp codecs

* fix: remove unused code

* feat: use array for multicodec fields

* fix: add timestamp if missing in v3 rpc

* fix: resolve on either lp codec when waiting for peer

* fix: remove unused util

* fix: remove unnecessary abstraction

* feat: accept nwaku docker image as arg, test lp backwards compat

* fix: revert filter error

* feat: add legacy flag to enable lightpushv2 only

* Revert "feat: accept nwaku docker image as arg, test lp backwards compat"

This reverts commit 857e12cbc73305e5c51abd057665bd34708b2737.

* fix: remove unused test

* feat: improve lp3 (#2597)

* improve light push core

* move back to singualar multicodec property, enable array prop only for light push

* implement v2/v3 interop e2e test, re-add useLegacy flag, ensure e2e runs for v2 and v3

* fix v2 v3 condition

* generate message package earlier

* add log, fix condition

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: Sasha <oleksandr@status.im>
2025-09-05 00:52:37 +02:00
fryorcraken
dc5155056b
feat!: SDS improvements and fixes (#2539)
* introduce `MessageId` type

# Conflicts:
#	packages/sds/src/message_channel/message_channel.ts

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* doc: long term solution is SDS protocol change

* SDS: renaming to match message function

* SDS: introduce `Message` class for easier encoding/decoding

# Conflicts:
#	packages/sds/src/message_channel/events.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS Message is a class now

* SDS: it's "possibly" not "partially" acknowledged.

* SDS: TODO

* SDS: fix tests

* SDS: make logs start with `waku`

* SDS: add bloom filter test

# Conflicts:
#	packages/sds/src/message_channel/events.spec.ts

* SDS: improve naming

* SDS: improve naming

Messages are not "sent" or received, but pushed for processing in local queues.

* SDS: sync message should not be delivered

* SDS: renaming from earlier

* SDS: remove useless variable

* SDS: Fix comment

* SDS: sync messages do not get "delivered"

* SDS: acks

* SDS: simplify delivered event

* SDS: improve event naming

* SDS: fix comment

* SDS: make task error an official event

* SDS: Mark messages that are irretrievably lost

* SDS: remove default for irretrievable and simplify config

* SDS: typo on sync event

* SDS: add and user sender id

* SDS: resent message never get ack'd

* SDS: fix cylic dependencies

* SDS: helpful logs

* SDS: avoid duplicate history entries

* SDS: export options
2025-08-12 10:47:52 +10:00
Arseniy Klempner
94788d0dfc
feat(proto): add light push v3 definition 2025-07-14 08:39:41 -07:00
Arseniy Klempner
408be95a13
feat(sds): add retrieval hint to causal history 2025-04-22 07:45:05 -07:00
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00
Danish Arora
86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Arseniy Klempner
6990e6f4e0
fix: use correct error message for store rate limit 2024-04-24 15:00:26 -07:00
NagyZoltanPeter
aa3ce1fc40
Added missing values for HistoryError to be sync with nwaku 2024-04-09 15:11:20 +02:00
Danish Arora
8ec11b2cde
chore: bump protons (#1898)
* regen lockfile

* commit new proto code

* chore: upgrade protons-runtime to 5.4.0

* chore: have src/generated as a dedicated directory for generated proto code

* chore: ignore unused vars in proto

* chore: update lockfile

* fix: eslint errors

* chore: handle flaky tests edge case

* chore: update playright docker image

* rm: only

* rm: console

* chore: regen lockfile

* chore: update playright docker image
2024-04-07 10:55:09 +03:00
Danish Arora
9ac2a3f363
feat: metadata protocol (#1732)
* add proto

* add rpc and interfaces

* add protocol implementation

* update faulty proto def

* add rpc and interfaces

* refactor implementation & write test

* setup the metadata protocol as a service

* fix cases where metadata service needs to be undefined

* remove redundant catch block

* remove addressed TODO

* update import path

* log errors

* remove redundant code from handling incoming metadata request

* update tests

* add test to check for active connections

* change expects

* save remote peer's shard info after successful connection
2023-12-05 19:26:52 +05:30
Danish Arora
78f64f60ee
chore: remove prettier, and default to eslint (#1495)
* rm: unwanted commit ff3ffdd138

* prettier: readds trailing commas

* update: .vscode settings to use eslint by default

* run eslint -- removes trailing commas

* remove: prettier

* remove linting from autogen proto

* address comments

* lint .eslintrc.json
2023-09-04 12:37:38 +05:30
Danish Arora
87717981eb
chore: upgrade libp2p and related deps (#1482)
* chore: update noise

* update: package.lock

* update: @chainsafe/libp2p-gossipsub

* rm unwanted libp2p interface deps & bump up libp2p

* refactor code for new deps

* update: new package.lock

* setup prettier, refactor eslint  and rm trailing commas

* update package.lock

* fix build

* import type for interface

* fix imports for merge

* update typedoc exports

* add: CustomEvent import

* use new libp2p interface

* add aegir as dev dep for tests
2023-08-16 20:18:13 +05:30
Sasha
51f9261a16
chore: update dependencies (#1460)
* update to latest prettier

* fix with prettier: added trailing comma

* remove deps from test package, add sinon types, fix type hack in a test, update esling prettier config

* update typescript eslint plugins

* update package-locks
2023-08-11 15:14:02 +02:00
Danish Arora
8d0e647966
feat!: filter v2 (#1332)
* implement proto

* implement filter v2

* add tests

* minor improvements
- make unsubscribe functions private in
filter
- enable all tests

* enable all tests

* readd multiaddrinput

* address comment removals

* unsubscribe based on contentFilters passed

* update unsubscribe function parameters in test

* reset interfaces & filter v1

* refactor filterv2 into 2 classes
- removes generics from types on filter which means
manual typecasting to filter version is required on
consumer side
- defaults to filterv2
- splits filterv2 into 2 classes:
  - one to create the subscription object with a
peer which returns the second class
  - the other to manage all subscription functions

* updates filter tests for the new API
- also fixes an interface import

* update `toAsyncIterator` test for Filter V1

* implement IReceiver on FilterV2

* remove return values from subscription functions

* update `to_async_iterator`

* address variable naming

* add tsdoc comments for hidden function

* address minor comments

* update docs to default to filter v2

* address comments

* rename `wakuFilter` to `wakuFilterV1`

* chore: Remove static variables (#1371)

* chore: Remove static variables

- Remove internal types from `@core/interfaces`
- Remove data being redundantly stored (pubsub topic)
- Remove usage of static variables
- Clean up callbacks and decoders when using `unsubscribe`
- Clean up callbacks and decoders when using `unsubscribeAll`

* fix setting activeSubscription

---------

Co-authored-by: danisharora099 <danisharora099@gmail.com>

* make activeSub getter and setter private

* update size-limit

---------

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2023-05-23 16:06:46 +05:30
fryorcraken.eth
bd983ea48e
feat!: enable encoding of meta field 2023-03-14 11:21:09 +11:00
fryorcraken.eth
967e6ffc7e
chore!: update store.proto
Ref: https://github.com/vacp2p/waku
2023-02-27 14:00:33 +11:00
fryorcraken.eth
5cf8ed2030
chore!: update message.proto: payload and content topic are always defined
Ref: https://github.com/vacp2p/waku
2023-02-27 14:00:33 +11:00
fryorcraken.eth
3e93e8f49e
chore: update light_push.proto
Ref: https://github.com/vacp2p/waku/blob/main/waku/lightpush/v2beta1/lightpush.proto
2023-02-27 14:00:32 +11:00
fryorcraken.eth
5f0e8b72f5
chore: update filter.proto
Ref: https://github.com/vacp2p/waku
2023-02-27 14:00:31 +11:00
fryorcraken.eth
2f2d266e81
chore!: bump protons from 5.1.0 to 7.0.2
Notable change: optionality of singular fields and optional values.

See https://github.com/ipfs/protons/releases
2023-02-10 22:26:46 +11:00
Danish Arora
0ca7fd790c
feat!: implement peer exchange (#1027)
* wip -- yet to test

* update: draft

* wip

* support passing flags manually to nwaku node

* refactor peer-exchange test

* switch response from uint8array to ENR

* rm: unnecesary logs

* implement clas

* fix: for loop

* init-wip: directories

* setup: new package & fix circular deps

* bind a response handler

* wip: refactor & update test

* test logs

* wip code - debugging

* address: comments

* Update packages/core/src/lib/waku_peer_exchange/peer_discovery.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/core/src/lib/waku_peer_exchange/peer_discovery.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address: comments

* address: comments

* address: comments

* address: comments

* address: comments

* fix: test build

* refactor

* fix: build

* comply with API

* numPeers: use number instead of bigint

* fix: build

* Update packages/peer-exchange/package.json

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/waku_peer_exchange.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/waku_peer_exchange.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/waku_peer_exchange.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address: comments, add eslint config

* Update packages/peer-exchange/.eslintrc.cjs

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* Update packages/peer-exchange/src/index.ts

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address comments

* test works with test fleet

* rm: only for px test => run all tests

* fix: tests

* reorder packages for build, and fix imports

* remove: px test doesnt work with local nodes

* chore: move proto into a separate package

* fix: proto dir

* fix: build

* fix: ci

* add: index for proto

* fix: ci

* Update packages/proto/package.json

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

* address comments

* chore: run failing test with higher timeout

* chore: run failing test with higher timeout

* fix: ci

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2022-12-07 11:35:30 +05:30
Danish Arora
56009d3b26
chore: move proto into a separate package (#1057)
* chore: move proto into a separate package

* fix: proto dir

* fix: build

* fix: ci

* add: index for proto

* fix: ci

* Update packages/proto/package.json

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2022-12-07 10:19:40 +05:30