37 Commits

Author SHA1 Message Date
Danish Arora
5ce36c8f18
feat!: deprecate named pubsub topics and use static/auto sharding (#2097)
* feat: deprecate named sharding & protocols adhere
simplify network config type, all protocols use pubsub topic internally

* chore: update tests

* tests: rm application info

* chore: use static sharding and auto sharding terminologies

* chore: update docs for network config

* chore: update interfaces

* tests: update tests error message

* chore: remove `ShardingParams` type and fix test
2024-08-13 05:23:20 +05:30
Danish Arora
86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Danish Arora
169a09d552
chore: enforce access modifiers (#2068)
* feat: introduce eslint flag

* chore: update logger

* chore: update enr

* chore: update core

* chore: update sdk

* chore: update relay

* chore: update discovery

* chore: update message-encryption

* chore: update tests

* chore: fix modifiers

* chore(tests): fix access modifiers

* chore: fix rebase
2024-07-19 15:58:17 +05:30
Danish Arora
e49e7289ae
chore: improve StreamManager (#1994)
* chore: improve stream manager

* chore: some cleaning

* chore: update API

* chore: rename `getConnectionStatus` to `isConnected`

* chore: use throw/catch

* chore: remove await

* remove redundant function and rename

* chore: increase test timeout
2024-05-14 16:31:38 +05:30
Danish Arora
bf42c8f53a
chore(store)!: move protocol implementation opinions to @waku/sdk (#1913)
* refactor the Store protocol into Core and SDK, simplify `queryGenerator()`

* update imports & types

* chore: `@noble/hashes` moves to `sdk`

* chore: update tests

* chore: update size-limit import path

* fix: cursor tests, use `Cursor` type from `proto.Index` instead of redefining

* export wakuStore from sdk

* fix: imports

* chore: use specific version for package

* chore: handle error for peer access

* use type instead of interface

* rm: comment

* add TODO

* chore!: remove deprecated function definition

* chore: improve logging
2024-04-01 16:47:47 +05:30
Arseniy Klempner
e138b4f5c4
feat: decouple sharding params out of core 2024-02-08 08:42:47 -08:00
Danish Arora
477c2a5918
feat!: protocols filter peers as per configured shard (#1756)
* merge: master

* fix: tests

* update: interfafces

* rm: comments

* metadata: store peerIdStr instead of peerId

* chore(utils): move fast-utils to dev deps

* fix: allow autosharding nodes to get peers (#1785)

* fix: merge

* fix: build

* fix: failing tests from master merge

---------

Co-authored-by: Arseniy Klempner <arseniyk@status.im>
2024-01-19 20:42:52 +05:30
Arseniy Klempner
ae7bc3eeec
chore: bump all libp2p packages to latest version 2024-01-17 11:59:08 -08:00
Danish Arora
b99f828cfb
chore: minor refactoring for protocols (#1762)
* remove: unnecessary function

* remove: test

* update doc

* add: tests

* revert unintended change

* fix: ping return

* fix(tests): stub connections

* log warning instead of throwing an error
2024-01-11 17:25:47 +05:30
Arseniy Klempner
69406bf1f0
reintroduce and deprecate named sharding (#1751)
Co-authored-by: danisharora099 <danisharora099@gmail.com>
2024-01-10 13:04:30 +05:30
Danish Arora
4cf2ffefa7
feat!: add support for sharded pubsub topics & remove support for named pubsub topics (#1697)
* merge branches

* tests: use a generator for sharded pubsub topics

* fix namespace edge case

* move shardInfo to pubsubTopic logic in waku.ts

* simplify encoder/decoder creation logic + update tests

* sharding utils: add error handling

* remove redundant test util

* baseprotocol: create abstraction for initialising pubsub topics

* fix: `createDecoder` interface

* filter: createSubscription takes shardInfo instead of pubsubTopicStr

* fix: sharding utils for error handling

* SingleShardInfo: use a new interface instead of omitting and rename namespace

* change redundant namespaces
2023-11-28 15:57:18 +05:30
Danish Arora
3166a5135e
chore!: change all instances of PubSubTopic to PubsubTopic (#1703)
* rename all PubSub patterns

* feat: forbid identifiers with camelcase pubSub (#1709)

---------

Co-authored-by: Arseniy Klempner <adklempner@gmail.com>
2023-11-14 18:52:52 +03:00
Danish Arora
0f7d63ef93
feat: Logger with log levels (#1672)
* setup a custom Logger with log level support

* refactor codebase for to use new Logger with log levels

* disallow usage of `debug` directly / only allow usage in/through custom Logger

* remove `debug` from logger
2023-10-20 16:36:47 +05:30
Danish Arora
9fe04d85f3
chore: use pubsubTopic/pubsubTopics across the codebase (#1653)
* chore: change all references of pubSubTopic to pubsubTopic

* change references of pubSubTopics to pubsubTopics

* flag words in cspell
2023-10-16 12:52:32 +05:30
Danish Arora
b10c46b910
fix(store)!: use pubSubTopic from DecodedMessage for createCursor (#1640)
* fix!(store): Cursor: use pubsubtopic from Message

* add control to check cursor topic should match decoder

* fix
2023-10-09 21:25:38 +05:30
Danish Arora
a3c45b6e1a
feat(static-sharding)!: allow multiple pubSubTopics (#1586)
* `ProtocolCreateOptions` now has `pubSubTopic` as `pubSubTopic[]`

* chore: update encoder & decoder to support `PubSubTopic`

* feat(protocols): allow multiple `PubSubTopic[]`

* feat(relay): allow multiple `PubSubTopic[]`

* chore(tests): update for new API

* chore: minor fixes

* chore: make store more robust

* fix(relay): correctly set types

* chore(address comments): update terminology around configured pubsub topics

* chore(address comments): minor refactoring

* chore(relay): split `subscribe` into smaller functions for readability & modularity

* chore(address comments): refactor `waitForGossipSubPeerInMesh`

* chore(store): only allow to query one `pubSubTopic`

* fix: `store` bug

* feat(tests): add some basic tests

* sharding utils

* address comments

* feat(relay): re-add API for `getMeshPeers`

* update error message

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* refactor for new API

* feat: simplify handling of observers (#1614)

* refactor: simplify handling of observers

* refactor: Remove redundant PubSubTopic from Observer

* use `??` instead of `||`

* update `pubsubTopic` to `pubSubTopic`

* update `interval` typo

* change occurence of `pubsubTopic` to `pubSubTopic`

* relay: rm `getAllMeshPeers` and make `pubSubTopics` public

* relay: use `push_or_init_map` and move to `utils`

* fix: update API for tests

* fix: relay waitForRemotePeer

---------

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2023-09-27 15:28:07 +05:30
Danish Arora
81a52a8097
feat!: set peer-exchange with default bootstrap (#1469)
* set peer-exchange with default bootstrap

* only initialise protocols with bootstrap peers

* update package

* update package-lock

* refactor `getPeers` while setting up a protocol

* move codecs to `@waku/interfaces`

* lightpush: send messages to multiple peers

* only use multiple peers for LP and Filter

* fix: ts warnings

* lightpush: tests pass

* update breaking changes for new API

* move codecs back into protocol files

* refactor: `getPeers()`

* rm: log as an arg

* add tsdoc for getPeers

* add import

* add prettier rule to eslint

* add: peer exchange to sdk as a dep

* fix eslint error

* add try catch

* revert unecessary diff

* revert unecessary diff

* fix imports

* convert relaycodecs to array

* remove: peerId as an arg for protocol methods

* keep peerId as an arg for peer-exchange

* remove: peerId from getPeers()

* lightpush: extract hardcoded numPeers as a constant

* return all peers if numPeers is 0 and increase readability for random peers

* refactor considering more than 1 bootstrap peers can exist

* use `getPeers`

* change arg for `getPeers` to object

* address comments

* refactor tests for new API

* lightpush: make constant the class variable

* use `maxBootstrapPeers` instead of `includeBootstrap`

* refactor protocols for new API

* add tests for `getPeers`

* skip getPeers test

* rm: only from test

* move tests to `base_protocol.spec.ts`

* break down `getPeers` into a `filter` method

* return all bootstrap peers if arg is 0

* refactor test without stubbing

* address comments

* update test title

* move `filterPeers` to a separate file

* address comments & add more test

* make test title more verbose

* address comments

* remove ProtocolOptions

* chore: refactor tests for new API

* add defaults for getPeers

* address comments

* rm unneeded comment

* address comment: add diversity of node tags to test

* address comments

* fix: imports
2023-09-07 13:15:49 +05:30
Danish Arora
b4f8216761
feat: pre-emptive stream creation for protocols (#1516)
* pass log as an arg to baseprotocol

* optimistically create and use streams for light protocols

* refactor BaseProtocol for readability

* use optimistic stream selection in protocols

* use a new stream for every request instead of reusing

* replenish streams correctly

* create StreamManager

* refactor for a single stream

* fix: listener binds

* declare streamManager as a class var isntead of extending

* remove stream destruction as it happens by default

* simplify logic & address comments

* fix: bind typo

* refactor for improvements

* fix typedoc

* rm: lock

* restructure StreamManager for readbility

* remove log as an arg

* use newStream as a facade in BaseProtoocl
2023-09-04 10:27:25 +05:30
Danish Arora
2389977a98
chore!: refactor store protocol for readability (#1456)
* refactor store protocol for readability

* update interface

* fix: test

* rm: comments

* Update packages/core/src/lib/store/index.ts

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* fix commit

---------

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2023-08-28 13:19:47 +05:30
Danish Arora
87717981eb
chore: upgrade libp2p and related deps (#1482)
* chore: update noise

* update: package.lock

* update: @chainsafe/libp2p-gossipsub

* rm unwanted libp2p interface deps & bump up libp2p

* refactor code for new deps

* update: new package.lock

* setup prettier, refactor eslint  and rm trailing commas

* update package.lock

* fix build

* import type for interface

* fix imports for merge

* update typedoc exports

* add: CustomEvent import

* use new libp2p interface

* add aegir as dev dep for tests
2023-08-16 20:18:13 +05:30
Sasha
51f9261a16
chore: update dependencies (#1460)
* update to latest prettier

* fix with prettier: added trailing comma

* remove deps from test package, add sinon types, fix type hack in a test, update esling prettier config

* update typescript eslint plugins

* update package-locks
2023-08-11 15:14:02 +02:00
Sasha
420e6c698d
feat!: upgrade to libp2p@0.45 (#1400)
* upgrade libp2p version, partially update protocols, rename to IBaseProtocol

* complete transition for protocols

* complete transition of connection maanger

* finish sdk

* complete core

* complete relay

* complete peer-exchange

* complete dns-discovery

* add components field to Libp2p interface and use it in core

* add type hack for Libp2p creation:

* finish waku node test

* complete relay test

* complete peer exchange

* complete dns peer discovery test

* add missing dependency to relay

* fix new peer store integration

* improve initialization of pubsub

* add catch for missing peer

* update test and remove extra dependency

* prevent error throw

* fix edge case with peerStore

* fix peer exchange

* fix protocols used

* fix test with another evnet

* bump libp2p and interfaces

* add missing package

* fix peer-exchange problem

* prefer libp2p peerDiscovery for integration tests

* fix import

* increate timeout

* return test against Test fleet

* remove await for peer:update

* increase timeout

* add await for peerStore

* comment event for testing

* fix lint

* remove bind

* fix stub

* decouple to separate test case

* move back to explicit build

* remove only

* do not test event
2023-07-25 02:17:52 +02:00
dependabot[bot]
df8e68ff36
chore(deps)(deps): bump it-length-prefixed from 8.0.4 to 9.0.1
Bumps [it-length-prefixed](https://github.com/alanshaw/it-length-prefixed) from 8.0.4 to 9.0.1.
- [Release notes](https://github.com/alanshaw/it-length-prefixed/releases)
- [Changelog](https://github.com/alanshaw/it-length-prefixed/blob/master/CHANGELOG.md)
- [Commits](https://github.com/alanshaw/it-length-prefixed/compare/v8.0.4...v9.0.1)

---
updated-dependencies:
- dependency-name: it-length-prefixed
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-06 14:07:05 +10:00
fryorcraken.eth
3a92355543
chore: code improvement
- Replace exception pattern with a result pattern as it better fits the
usage.
- merge few maps to prefer chaining to assign var after var.
- Make `isDefined` type helper a common util function
2023-03-21 08:59:25 +11:00
Danish Arora
a30b2bd747
chore!: add exports map to @waku/utils (#1201)
* add exports map

* merge: master

* export  subdir with default export path

* import according to the export path
2023-03-14 10:10:38 +05:30
fryorcraken.eth
628ac50d71
feat!: expose pubsub topic in IDecodedMessage
Needed for deterministic message hash.

Ref: #1208
2023-03-13 13:49:24 +11:00
fryorcraken.eth
1a9b13f902
chore: use BaseProtocol on WakuPeerExchange
Ref: https://github.com/waku-org/js-waku/pull/1137
2023-02-28 13:49:36 +11:00
fryorcraken.eth
81d6638067
chore: use pascal case for class name 2023-02-27 14:00:49 +11:00
fryorcraken.eth
967e6ffc7e
chore!: update store.proto
Ref: https://github.com/vacp2p/waku
2023-02-27 14:00:33 +11:00
fryorcraken.eth
789bebed00
chore: add BaseProtocol class to req-resp protocols
Each protocol implementation have some common functionality.
Implements those in `BaseProtocol`, to be extended by each protocol
implementation.
2023-02-27 09:48:30 +11:00
fryorcraken.eth
c85b113df7
chore: use this.multicodec over constant
To enable extraction of common functions.
2023-02-27 09:48:10 +11:00
fryorcraken.eth
803ae7bd8e
chore!: bump libp2p dependencies 2023-02-09 19:30:59 +11:00
Danish Arora
b33306655a
chore: move libp2p and byte utils to a single utils package (#1149)
* move libp2p and byte utils to a single `utils`
package

* set package version to 0.0.1
2023-02-06 11:22:36 +05:30
Danish Arora
10b3898762
chore: move protocols CreateOptions into interfaces (#1145)
* move protocols `CreateOptions` into interfaces and
add possible TODO

* remove: createOptions for PeerExchange

* update test with new API
2023-02-02 08:02:06 +05:30
fryorcraken.eth
a5ff788eed
feat: codec as a property of the protocol implementations
For easy access from `@waku/core` without the need to depend on package
implementing the protocol.
2023-01-27 09:08:02 +11:00
fryorcraken.eth
d946e443ad
refactor: remove Waku from protocol names
This is redundant with the context (waku package).
2022-12-12 22:36:13 +11:00
fryorcraken.eth
574f3ee80f
refactor: remove waku_ from module names
This is redundant with the context (waku package).
2022-12-12 22:36:13 +11:00