44 Commits

Author SHA1 Message Date
Sasha
1905558753
feat!: re-work messaging parts and sharding (#2399)
* remove message-hash package, move it to core

* add js-doc to message hash

* up

* address type changes

* fix lint
2025-06-03 11:08:02 +02:00
Sasha
3136f3a704
feat: add HealthIndicator with simplified logic and testing (#2251)
* implement HealthIndicator

* up libp2p interface version

* up lock

* remove unused tests

* expose HealthIndicator from Waku

* update test, add start and stop

* fix error handling
2025-02-05 13:24:50 +01:00
Sasha
62f93dc842
feat: improve peer manager and re-integrate to light push (#2191)
* up lock

* make ConnectionManager use ctor

* reform connection manager configurations

* remove log param from peerManager

* make PeerManager use only ConnectionManager, move getPeers to ConnectionManager, remove not needed code

* remove allPeers and connectedPeers from BaseProtocolCore, update tests, add getPeers for IWaku

* use only one peerManager from Waku object

* remove IBaseProtocolSDK and merge with PeerManager

* re-implement peerManager, remove ProtocolUseOptions

* remove not needed test, up lock

* update deps and lock

* remove old test for peerManager, fix check and spell

* rename to getConnectedPeers

* feat: improve filter subscriptions (#2193)

* add message cache to Filter

* remove WakuOptions and use only ProtocolCreateOptions

* move subscribe options to createLightNode Fitler protocol options

* rename SubscriptionManager to Subscription

* rename to CreateNodeOptions

* add warning

* feat: introduce subscription manager (#2202)

* feat: inroduce subscription manager

* fix: make pipeline succeed (#2238)

* fix test

* use hardcoded value

* update playwright

* fix test:browser

* up lock

* make peer retrieval probabilistic

* add comments

* up lightpush tests

* add tests for peer_manager, improve folder structure

* create named files for protocols

* create named files, simplify project structure

* remove only
2025-01-31 00:16:00 +01:00
Sasha
1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Danish Arora
86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Danish Arora
d464af3645
feat: node and protocols health (#2080)
* feat: introduce HealthManager

* feat: make health accessible on Waku object

* feat: update health from protocols

* chore: add access modifiers to healthmanager

* feat: use a HealthManager singleton

* chore: add tests for Filter, LightPush and Store

* feat: add overall node health

* chore: update protocol health to consider Store protocol

* chore: setup generic test utils instead of using filter utils

* tests: add a health status matrix check from 0-3

* chore: increase timeout for failing tests in CI
tests pass locally without an increased timeout, but fail in CI

* chore: move name inference to HealthManager

* tests: abstract away node creation and teardown utils

* fix: import
2024-07-27 18:27:54 +05:30
Sasha
00635b7afe
feat: fix peer renewal, change Filter keep alive (#2065)
* move util, create stream manager folder

* change default keep alive, improve peer renewal

* address nit
2024-07-16 18:35:24 +02:00
Danish Arora
5fb100602b
chore(lightpush)!: move protocol implementation to @waku/sdk (1/n) (#1964)
* chore: decouple `Filter` between `core` and `sdk`
moves `SubscriptionManager` to `sdk` side

* chore: update package dependencies
also update peer deps in sdk

* chore: update imports

* chore: update tests

* chore(side-change): update lightpush

* chore: update size-limit import

* chore(sdk): update dependencies
2024-04-19 17:20:34 +05:30
Danish Arora
bf42c8f53a
chore(store)!: move protocol implementation opinions to @waku/sdk (#1913)
* refactor the Store protocol into Core and SDK, simplify `queryGenerator()`

* update imports & types

* chore: `@noble/hashes` moves to `sdk`

* chore: update tests

* chore: update size-limit import path

* fix: cursor tests, use `Cursor` type from `proto.Index` instead of redefining

* export wakuStore from sdk

* fix: imports

* chore: use specific version for package

* chore: handle error for peer access

* use type instead of interface

* rm: comment

* add TODO

* chore!: remove deprecated function definition

* chore: improve logging
2024-04-01 16:47:47 +05:30
Danish Arora
8deab11890
chore(lightpush)!: move protocol implementation opinions to @waku/sdk (#1887)
* chore: restructure @waku/sdk

* chore: introduce `BaseProtocolCore` and `BaseProtocolSDK`

* chore: introduce `LightPushCore` and `LightPushSDK`

* chore: update `relay` for new types

* chore(sdk): update structure

* chore(filter): add `numPeersToUse`

* chore: update tests

* update: size-limit

* chore: update more tests

* attach issue link to TODOs
2024-03-11 18:50:34 +05:30
Arseniy Klempner
ee2d4176f8
feat: create node and subscription by content topic 2024-02-27 15:37:57 -08:00
Danish Arora
477c2a5918
feat!: protocols filter peers as per configured shard (#1756)
* merge: master

* fix: tests

* update: interfafces

* rm: comments

* metadata: store peerIdStr instead of peerId

* chore(utils): move fast-utils to dev deps

* fix: allow autosharding nodes to get peers (#1785)

* fix: merge

* fix: build

* fix: failing tests from master merge

---------

Co-authored-by: Arseniy Klempner <arseniyk@status.im>
2024-01-19 20:42:52 +05:30
Danish Arora
9983549e8b
chore: use connected peers instead of discovered peers for protocols (#1758)
* switch to use connected peers instead of discovered peers for protocols

* switch to use connected peers instead of discovered peers for protocols
2024-01-02 15:49:31 +05:30
danisharora099
2bc3735e4d
feat: add support for autosharded pubsub topics
tests: use a generator for sharded pubsub topics
set pubsub topic in encoder/decoder based on sharding type
add function for grouping content topics by pubsub topic
add autosharding config to create options
add autoshard rpc endpoints to nwaku and use in tests
set autoshard pubsub topics in all protocols
fix rebase with static sharding
removes unused function
remove console logs
remove autosharding from ShardInfo, add to EncoderOptions
fix enr and encoder/decoder options
test that same application/version hashes to same shard index
update comment on shard field
fix spelling of autosharding
fix content topic protocol in tests
add sharding type alias and function to determine topic in encoders/decoders
move DefaultPubsubTopic from core to interfaces
2023-12-21 10:03:22 -08:00
Danish Arora
9ac2a3f363
feat: metadata protocol (#1732)
* add proto

* add rpc and interfaces

* add protocol implementation

* update faulty proto def

* add rpc and interfaces

* refactor implementation & write test

* setup the metadata protocol as a service

* fix cases where metadata service needs to be undefined

* remove redundant catch block

* remove addressed TODO

* update import path

* log errors

* remove redundant code from handling incoming metadata request

* update tests

* add test to check for active connections

* change expects

* save remote peer's shard info after successful connection
2023-12-05 19:26:52 +05:30
Danish Arora
3166a5135e
chore!: change all instances of PubSubTopic to PubsubTopic (#1703)
* rename all PubSub patterns

* feat: forbid identifiers with camelcase pubSub (#1709)

---------

Co-authored-by: Arseniy Klempner <adklempner@gmail.com>
2023-11-14 18:52:52 +03:00
Danish Arora
81a52a8097
feat!: set peer-exchange with default bootstrap (#1469)
* set peer-exchange with default bootstrap

* only initialise protocols with bootstrap peers

* update package

* update package-lock

* refactor `getPeers` while setting up a protocol

* move codecs to `@waku/interfaces`

* lightpush: send messages to multiple peers

* only use multiple peers for LP and Filter

* fix: ts warnings

* lightpush: tests pass

* update breaking changes for new API

* move codecs back into protocol files

* refactor: `getPeers()`

* rm: log as an arg

* add tsdoc for getPeers

* add import

* add prettier rule to eslint

* add: peer exchange to sdk as a dep

* fix eslint error

* add try catch

* revert unecessary diff

* revert unecessary diff

* fix imports

* convert relaycodecs to array

* remove: peerId as an arg for protocol methods

* keep peerId as an arg for peer-exchange

* remove: peerId from getPeers()

* lightpush: extract hardcoded numPeers as a constant

* return all peers if numPeers is 0 and increase readability for random peers

* refactor considering more than 1 bootstrap peers can exist

* use `getPeers`

* change arg for `getPeers` to object

* address comments

* refactor tests for new API

* lightpush: make constant the class variable

* use `maxBootstrapPeers` instead of `includeBootstrap`

* refactor protocols for new API

* add tests for `getPeers`

* skip getPeers test

* rm: only from test

* move tests to `base_protocol.spec.ts`

* break down `getPeers` into a `filter` method

* return all bootstrap peers if arg is 0

* refactor test without stubbing

* address comments

* update test title

* move `filterPeers` to a separate file

* address comments & add more test

* make test title more verbose

* address comments

* remove ProtocolOptions

* chore: refactor tests for new API

* add defaults for getPeers

* address comments

* rm unneeded comment

* address comment: add diversity of node tags to test

* address comments

* fix: imports
2023-09-07 13:15:49 +05:30
Danish Arora
9fc79f6853
chore: export FilterCodecs (#1532)
* export FilterCodecs

* root export
2023-09-05 15:38:02 +05:30
Danish Arora
b4f8216761
feat: pre-emptive stream creation for protocols (#1516)
* pass log as an arg to baseprotocol

* optimistically create and use streams for light protocols

* refactor BaseProtocol for readability

* use optimistic stream selection in protocols

* use a new stream for every request instead of reusing

* replenish streams correctly

* create StreamManager

* refactor for a single stream

* fix: listener binds

* declare streamManager as a class var isntead of extending

* remove stream destruction as it happens by default

* simplify logic & address comments

* fix: bind typo

* refactor for improvements

* fix typedoc

* rm: lock

* restructure StreamManager for readbility

* remove log as an arg

* use newStream as a facade in BaseProtoocl
2023-09-04 10:27:25 +05:30
Danish Arora
87717981eb
chore: upgrade libp2p and related deps (#1482)
* chore: update noise

* update: package.lock

* update: @chainsafe/libp2p-gossipsub

* rm unwanted libp2p interface deps & bump up libp2p

* refactor code for new deps

* update: new package.lock

* setup prettier, refactor eslint  and rm trailing commas

* update package.lock

* fix build

* import type for interface

* fix imports for merge

* update typedoc exports

* add: CustomEvent import

* use new libp2p interface

* add aegir as dev dep for tests
2023-08-16 20:18:13 +05:30
Danish Arora
0b8936f1f1
feat: ConnectionManager extends EventEmitter & exposed on the Waku interface (& minor improvements) (#1447)
* move KeepAliveOptions to dedicated interface file

* update export for KeepAlive

* expose `ConnectionManager` on the waku node

* update ConnectionManager test to use the exposed API

* rm: only for the test
2023-07-31 13:54:39 +05:30
Danish Arora
d483644a4b
chore!: remove filter v1 (#1433)
* rm: v1

* fix v2 imports

* remove tests for filter v1

* set filter v1 as default and rm v2 completely

* change import name for filter v2

* rename FilterV2 to Filter completely

* fix run check
2023-07-26 11:30:48 +05:30
Danish Arora
8d0e647966
feat!: filter v2 (#1332)
* implement proto

* implement filter v2

* add tests

* minor improvements
- make unsubscribe functions private in
filter
- enable all tests

* enable all tests

* readd multiaddrinput

* address comment removals

* unsubscribe based on contentFilters passed

* update unsubscribe function parameters in test

* reset interfaces & filter v1

* refactor filterv2 into 2 classes
- removes generics from types on filter which means
manual typecasting to filter version is required on
consumer side
- defaults to filterv2
- splits filterv2 into 2 classes:
  - one to create the subscription object with a
peer which returns the second class
  - the other to manage all subscription functions

* updates filter tests for the new API
- also fixes an interface import

* update `toAsyncIterator` test for Filter V1

* implement IReceiver on FilterV2

* remove return values from subscription functions

* update `to_async_iterator`

* address variable naming

* add tsdoc comments for hidden function

* address minor comments

* update docs to default to filter v2

* address comments

* rename `wakuFilter` to `wakuFilterV1`

* chore: Remove static variables (#1371)

* chore: Remove static variables

- Remove internal types from `@core/interfaces`
- Remove data being redundantly stored (pubsub topic)
- Remove usage of static variables
- Clean up callbacks and decoders when using `unsubscribe`
- Clean up callbacks and decoders when using `unsubscribeAll`

* fix setting activeSubscription

---------

Co-authored-by: danisharora099 <danisharora099@gmail.com>

* make activeSub getter and setter private

* update size-limit

---------

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2023-05-23 16:06:46 +05:30
Danish Arora
50c2c2540f
feat!: @waku/relay (#1316)
* move constants & core's func to @waku/utils

* setup @waku/relay

* use @waku/relay for create

* address: principal review

* fix: size-limit

* move all constants to @waku/core

* move TopicOnlyDecoder to relay

* merge: master

* fix: tests

* Move constants out of core (#1340)

* update package.lock

* change constants folder into a ts file

* use dependency version as * instead of version number

* add constants import from @waku/core

* add empty changelog file for relay

---------

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2023-05-11 14:08:00 +05:30
Sasha
e11e5b4870
feat!: add and implement IReceiver (#1219)
- remove extend Relay by GossipSub and use it as public property;
- detach GossipSub initialisation;
2023-03-31 03:17:41 +02:00
fryorcraken.eth
da1b18d995
feat: export Decoder, Encoder and DecodedMessage types from root
Fixes #1010
2023-03-08 09:52:27 +11:00
Danish Arora
24c24cc27d
feat!: ConnectionManager and KeepAliveManager (#1135)
* initialises ConnectionManager and KeepAliveManager
ports from previous PR and makes necessary improvements and reductions

* address: comments

* map a ConnectionManager instance with a WakuNode

* abstract event listeners logic

* minor fix

* minor cleaning

* instantiate KeepAliveManager instead of extending

* fix build and enable all tests

* fix CI

* address review

* refine event handlers
- only removes the previously attached callback from
the event handlers while shutting down service
- removes the requirement of passing around
`keepAliveOptions` and `relay` inside of
`ConnectionManager`

* add verbosity to interface

* make `dialPeer()` more readable

* use set to push tags to avoid duplicates

* fix: merge build

* remove: logging function

* rename startService and stopService

* remove: future TODO

added that as part of future refactor

* use the new libp2p api

* initialise options in constructor//fix TS error

* remove stale export

* address principal review

* reset test timeout to master
2023-02-20 15:30:59 +05:30
fryorcraken.eth
ac4ff5a00a
chore: fix links in docs, treat doc warnings as errors 2022-12-21 16:47:00 +11:00
fryorcraken.eth
e0fd4433e7
fix: remove invalid re-export 2022-12-19 16:16:07 +11:00
fryorcraken.eth
574f3ee80f
refactor: remove waku_ from module names
This is redundant with the context (waku package).
2022-12-12 22:36:13 +11:00
Danish Arora
56009d3b26
chore: move proto into a separate package (#1057)
* chore: move proto into a separate package

* fix: proto dir

* fix: build

* fix: ci

* add: index for proto

* fix: ci

* Update packages/proto/package.json

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>

Co-authored-by: fryorcraken.eth <110212804+fryorcraken@users.noreply.github.com>
2022-12-07 10:19:40 +05:30
fryorcraken.eth
33ed43c068
feat: waitForRemotePeer can now be directly imported from @waku/core 2022-12-05 15:14:51 +11:00
fryorcraken.eth
e8efd5e962
feat!: export user functions from package root
`createEncoder`, `createDecoder` and `DecodedMessage` are function/types
useful to the user so they should have easy access to it.

We still export `Decoder` and `Encoder` but in a path so it cam be
re-used by `@waku/message-encryption`.
2022-12-05 09:34:17 +11:00
fryorcraken.eth
88c6ec6ef4
chore: add '.js' extensions to local import
As per ESM standard.
2022-12-02 15:54:30 +11:00
danisharora099
100eb4c4b4
Merge branch 'master' into danisharora/cursor-support-store 2022-11-17 13:00:48 +05:30
fryorcraken.eth
ee009742c3
Merge branch 'master' of github.com:waku-org/js-waku into danisharora/add-user-agent 2022-11-17 14:41:01 +11:00
fryorcraken.eth
a372307a30
refactor: adopt dependency injection patter for other Waku protocols
It actually simplies the API as the caller does not need to pass
libp2p to each protocol anymore (when not using a `create*` helper).
2022-11-17 12:06:43 +11:00
fryorcraken.eth
8dfb133cd7
fix: libp2p upgrade (no more autodial + constructor dependency injection) 2022-11-16 21:27:33 +11:00
danisharora099
0169a0ccb1
functionality works! test wip 2022-11-15 05:17:24 +05:30
danisharora099
7d8c410b5b
address comments 2022-11-14 15:13:16 +05:30
fryorcraken.eth
256b7223f3
chore!: extract version-1 from chore 2022-11-04 11:28:14 +11:00
fryorcraken.eth
21e5e41b0a
chore: build core package 2022-11-03 14:24:08 +11:00
fryorcraken.eth
057507f94f
fix: @waku/interfaces 2022-11-01 16:30:24 +11:00
fryorcraken.eth
b975943e90
chore: move js-waku to ./core 2022-10-30 17:57:15 +11:00