99 Commits

Author SHA1 Message Date
Sasha
75fcca4cd9
feat: replace waitForRemotePeers() with waku.waitForPeer() method (#2161)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* add I prefix to Waku interface

* implement waku.connect method

* add docs to IWaku interface

* remove export and usage of waitForRemotePeer

* move wait for remote peer related to Realy out of @waku/sdk

* change tests to use new API

* fix linting

* update size limit

* rename .connect to .waitForPeer

* export waitForRemotePeer and mark as deprecated

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer (#2163)

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer

* add waitForRemote UTs

* remove junk

* feat: expose peerId and protocols from WakuNode (#2166)

* chore: expose peerId and protocols from WakuNode

* remove unused method

* move to private method

* rename to waitForPeers

* up test
2024-10-09 00:43:34 +02:00
Sasha
d37e0245cf
feat: confirm metadata and protocols needed in waitForRemotePeer (#2160)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* imrpove iteration for existing connections

* address protocol adverisement in CI

* add protocols needed

* add missing protocols

* make lightpush and filter default for tests

* up
2024-10-05 01:49:54 +02:00
Sasha
1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Danish Arora
9d9a696024
chore: mandate .js extensions for imports (#2128)
* feat: add eslint rule

* chore: update codebase import extensions
2024-09-13 14:18:29 +05:30
Sasha
5cfe9327b6
chore: remove relay from sdk package (#2040)
* remove full node

* remove relay from sdk

* comment node counter check

* try using logline

* up comment
2024-08-31 15:18:51 +02:00
Sasha
71384dfdfd
chore: change amount of used peers to 2 (#2111)
* chore: change amount of used peers to 2

* up

* up

* up tests
2024-08-27 21:53:16 +02:00
Danish Arora
409642d7d3
chore: upgrade libp2p and related packages (#2084)
* chore: upgrade libp2p and related packages

* fix: public key on peer id can now be undefined

* chore: upgrade remainder packages

* chore: fix tests

* chore: address comments

* chore: upgrade playwright and proto
2024-08-14 04:23:04 +05:30
Danish Arora
5ce36c8f18
feat!: deprecate named pubsub topics and use static/auto sharding (#2097)
* feat: deprecate named sharding & protocols adhere
simplify network config type, all protocols use pubsub topic internally

* chore: update tests

* tests: rm application info

* chore: use static sharding and auto sharding terminologies

* chore: update docs for network config

* chore: update interfaces

* tests: update tests error message

* chore: remove `ShardingParams` type and fix test
2024-08-13 05:23:20 +05:30
Danish Arora
33a9172bbb
chore: upgrade nwaku to 0.31 (#2091)
* chore: upgrade nwaku to 0.31

* tests: fix tests by removing reliance on named sharding
2024-08-01 14:33:42 +05:30
Danish Arora
d464af3645
feat: node and protocols health (#2080)
* feat: introduce HealthManager

* feat: make health accessible on Waku object

* feat: update health from protocols

* chore: add access modifiers to healthmanager

* feat: use a HealthManager singleton

* chore: add tests for Filter, LightPush and Store

* feat: add overall node health

* chore: update protocol health to consider Store protocol

* chore: setup generic test utils instead of using filter utils

* tests: add a health status matrix check from 0-3

* chore: increase timeout for failing tests in CI
tests pass locally without an increased timeout, but fail in CI

* chore: move name inference to HealthManager

* tests: abstract away node creation and teardown utils

* fix: import
2024-07-27 18:27:54 +05:30
gabrielmer
08fc2d133a
chore: changing default pubsub topic to its static sharding version (#2083)
* changing default pubsub topic to its static sharding version

* keeping RFC's Waku Message test vectors

* reverting change in changelog

* setting pubsub topic when creating nwaku node

* adding shardInfo to runMultipleNodes call

* adding shardInfo to runMultipleNodes call in lightpush tests

* add pubsub topics to nwaku.start

* get rid of it.only that remained

* fixing compliance tests

* setting clusterId to 0

* removing unnecessary fix

* adding shardInfo when creating nodes

* fixing wait for remote peer tests

* fixing peer exchange test

* refactor

* removing unnecessary variable

* feat: create default shard info, update tests (#2085)

* feat: create default shard info, update tests

* add link

* fix tests

* remoe only

* up tests

* up test

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2024-07-26 00:21:52 +02:00
Danish Arora
169a09d552
chore: enforce access modifiers (#2068)
* feat: introduce eslint flag

* chore: update logger

* chore: update enr

* chore: update core

* chore: update sdk

* chore: update relay

* chore: update discovery

* chore: update message-encryption

* chore: update tests

* chore: fix modifiers

* chore(tests): fix access modifiers

* chore: fix rebase
2024-07-19 15:58:17 +05:30
Danish Arora
42126a6301
chore: upgrade nwaku to 0.29.0 (#2042) 2024-06-20 11:30:55 -04:00
Arseniy Klempner
aad819bb1f
feat: add rpc url to nwaku, persist rln tree in docker and ci 2024-05-13 07:42:08 -07:00
Danish Arora
4eb06c64eb
feat(filter)!: return error codes instead of throwing errors (#1971)
* move protocol result type to interfaces

* chore: update type names for verbosity

* feat(filter-core): convert error throws to return types

* chore: update types & imports

* update Filter API

* chore: update createSubscription

* chore: update imports & rename

* chore: update all tests

* chore: resolve conflicts & merge (2/n)

* chore: resolve conflicts & merge (3/n)

* chore: resolve conflicts & merge (4/n)

* chore: resolve conflicts & merge (5/n)

* chore: resolve conflicts & merge (6/n)

* chore: use idiomatic approach

* chore: fix tests

* chore: address comments

* chore: fix test

* rm: only
2024-05-09 16:51:08 +05:30
Arseniy Klempner
5df41b0adf
fix: add shard arg to nwaku in peer management tests (#2005) 2024-05-09 10:25:35 +02:00
Sasha
f3627c46a4
feat!: use ShardingParams on subscriptions, make Decoder/Encoder auto sharding friendly by default (#1958)
* fix: use pubsubTopic from current ones if not set

* fix: improve type on dial method

* enforce same pubusb on filter.subscribe, make content topic to pubsub mapping default for decoder / encoder

* fix mapping problem

* update tests

* add error handling

* fix typo

* up lock

* rm lock

* up lock

* remove only

* fix content topic

* fix ephemeral test

* fix filter unsubscribe test

* up utils

* fix subscribe test

* up interfaces and filter api

* remove only

* up ping test

* fix subscribe test

* fix push test

* fix lightPush

* fix multiple pubsub

* remove only, fix subscribe filter test

* remove only

* fix cluster ID selection and named sharding subscription test

* fix unsubscribe test

* fix light push test

* fix light push test

* fix push test

* fix relay publish

* create runNode and fix relay tests

* generalize runNodes, fix some tests

* fix store tests

* fix toAsyncIterator tests

* remove only

* fix lightPush

* use generics

* try fix test

* run failing tests

* remove only

* address failed tests, remove DefaultPubsubTopic dependency in some tests
2024-04-28 11:15:17 +02:00
fbarbu15
7ee02faf48
chore: use nwaku:v0.27.0 and adjust tests for it (#1975)
* use nwaku:v0.27.0 and adjust tests for it

* fix sharding tests
2024-04-24 21:30:55 +02:00
Florin Barbu
f4c7c02f83
test: sharding tests refactor (#1883)
* sharding tests refactor

* small fixes

* adjust clusterID based on version

* fix typo

* fix dispatchEvent test

* sharding unit tests

* port adjustment

* update unit tests

* fix 1902

* adjust content topic tests

* adjust metdata tests

* small adjustments

* fix

* update resolveAutoshardingCluster version

* skip autosharding tests for nwaku < 0.27.0

* skip autosharding tests for nwaku < 0.27.0
2024-03-18 13:40:08 +02:00
Danish Arora
639da72aad
chore: upgrade nwaku to v0.26 (#1906)
* chore: upgrade to v0.26

* skip TWN tests
2024-03-13 20:51:51 +05:30
Florin Barbu
36dfcc9560
test: fix dispatchEvent test (#1897)
* fix dispatchEvent test

* skip failing test
2024-03-10 04:10:33 +05:30
Arseniy Klempner
37f24e9ea1
fix: remove rpc settings when starting nwaku image 2024-03-04 07:16:16 -08:00
Sasha
9f198dd149
feat: add bootstrapPeers option and refactor sdk (#1871)
* move relay related code

* move libp2p to utils

* define CreateWakuNodeOptions

* improve options

* make libp2p use from create function

* add bootstrapPeers option

* fix lint

* fix types, imports

* fix exports

* use bootstrap along default bootstrap

* fix test as REST does not return peer though connection is made

* rollback condition change

* enable gossipSub back for every node
2024-03-04 10:56:20 +01:00
Florin Barbu
df8c0d79f7
test: peer exchange tests (#1859)
* peer exchange tests

* adjust after test with nwaku 25
2024-02-27 09:54:34 +02:00
Florin Barbu
7af6b71d24
upgrade nwaku to v0.25 (#1861) 2024-02-22 16:04:43 +02:00
Arseniy Klempner
6009af7453
feat: migrates e2e tests to use rest instead of rpc 2024-02-16 18:52:51 -08:00
Arseniy Klempner
739f2bc2c9
feat: switches to REST for calling nwaku messages/subscription endpoints
This commit modifies functions in ServiceNode to use the REST API
instead of JSON RPC when reading messages for a pubsub topic or
content topic, and when ensuring a nwaku node is subscribed to a
pubsub topic. Also modifies default Docker params to enable the
rest API and provide a port.
2024-02-15 14:39:40 -08:00
Florin Barbu
de5be4413b
chore: use graceful timeout mechanism (#1841)
* use graceful timeout mechanism

* set max timeout for all hooks

* small fix

* small fix

* use MOCHA_HOOK_MAX_TIMEOUT as default timeoutDuration

* force retry even when the hook fails

* use custom hooks

* fix global timeout problem

* fix unwanted change

* fix enr issue

* force retry on before error as well
2024-02-15 08:50:03 +02:00
Danish Arora
f8e02ab19e
fix(tests): append p2p with the multiaddrs from ENR (#1817)
* append `p2p` with the multiaddrs from ENR

* fix(tests): add p2p checks for ENR multiaddrs

* fix(tests): type getter

* chore(ci): remove debug flag from nwaku_master and go-waku tests
2024-02-07 01:57:41 +05:30
Florin Barbu
dbb51e4c86
test: fix for nwaku 24 (#1820)
* bump nwaku to v0.24.0

* fix tests on nwaku 0.24

* fix multiple service node filter sub tests

* fix timeout for metadata tests
2024-01-31 17:31:09 +02:00
Danish Arora
7affbe222d
feat: lightpush & filter send requests to multiple peers (#1779)
* feat: lightpush & filter send requests to multiple peers

* build message-hash before core

* chore: restructure folder heirrarchy

* fix: imports

* chore: move @waku/core to dev deps

* feat: create a new `ServiceNodes` wrapper class to encapsulate service node and message collector with redundancy accounted for

* chore(filter): move tests against single service node to a subdir

* feat: support relay, add strict checking, add tests

* fix(filter): handle errors

* chore(tests): add tests for ping

* add tests for push

* chore: abstract redundancy

* feat: add unsubscribe tests

* fix: tests

* add lightpush tests

* fix: imports

* fix: merge & add warning

* merge: master

* fix: breaking tests with master

* address comments

* make num peers configurable

* fix: typo
2024-01-24 18:24:03 +05:30
Florin Barbu
13d3d70609
chore: increase max sub topic size to 100 (#1791)
* increase max sub topic size to 100

* make tests run fine both ways

* fix:import error

* update nwaku and gowaku master images

* prepare for pr
2024-01-18 21:22:51 +05:30
Arseniy Klempner
ae7bc3eeec
chore: bump all libp2p packages to latest version 2024-01-17 11:59:08 -08:00
Danish Arora
2e6d9836bf
chore(tests): restructure & cleanup (#1796)
* chore: restructure folder heirrarchy

* fix: imports
2024-01-18 00:26:31 +05:30
Danish Arora
dc96074c81
fix(tests): peer-exchange interop (#1773)
* fix(tests): px interop

* optimize waitFor utility

* address minor comments

* fix: test refactoring
2024-01-11 16:39:48 +05:30
danisharora099
2bc3735e4d
feat: add support for autosharded pubsub topics
tests: use a generator for sharded pubsub topics
set pubsub topic in encoder/decoder based on sharding type
add function for grouping content topics by pubsub topic
add autosharding config to create options
add autoshard rpc endpoints to nwaku and use in tests
set autoshard pubsub topics in all protocols
fix rebase with static sharding
removes unused function
remove console logs
remove autosharding from ShardInfo, add to EncoderOptions
fix enr and encoder/decoder options
test that same application/version hashes to same shard index
update comment on shard field
fix spelling of autosharding
fix content topic protocol in tests
add sharding type alias and function to determine topic in encoders/decoders
move DefaultPubsubTopic from core to interfaces
2023-12-21 10:03:22 -08:00
Danish Arora
bbd372120a
chore: update all references of shard info to RFC terminology (#1740)
* update all references of shard info to RFC terminology

* bump nwaku to v0.22.0 (#1741)
2023-11-29 17:37:59 +05:30
Danish Arora
4d19fc7b75
bump nwaku to v0.22.0 (#1741) 2023-11-29 15:48:50 +05:30
Danish Arora
4cf2ffefa7
feat!: add support for sharded pubsub topics & remove support for named pubsub topics (#1697)
* merge branches

* tests: use a generator for sharded pubsub topics

* fix namespace edge case

* move shardInfo to pubsubTopic logic in waku.ts

* simplify encoder/decoder creation logic + update tests

* sharding utils: add error handling

* remove redundant test util

* baseprotocol: create abstraction for initialising pubsub topics

* fix: `createDecoder` interface

* filter: createSubscription takes shardInfo instead of pubsubTopicStr

* fix: sharding utils for error handling

* SingleShardInfo: use a new interface instead of omitting and rename namespace

* change redundant namespaces
2023-11-28 15:57:18 +05:30
Danish Arora
3166a5135e
chore!: change all instances of PubSubTopic to PubsubTopic (#1703)
* rename all PubSub patterns

* feat: forbid identifiers with camelcase pubSub (#1709)

---------

Co-authored-by: Arseniy Klempner <adklempner@gmail.com>
2023-11-14 18:52:52 +03:00
Florin Barbu
9593a8a8a5
test: fix ci logs (#1687)
* fix ci logs

* replace lodash with uint8arrays/equals
2023-10-25 10:39:04 +03:00
Florin Barbu
1ec0c200ca
chore: new relay tests (#1649)
* make relay folder

* make relay folder

* adjust message collector for relay

* small fix

* small fix

* small fix

* split tests more

* small fixes

* small fix

* new test

* fix pubsubtopic name

* new subscribe tests

* new subscribe tests

* new tests

* small fix after ci run

* small fix after ci run2

* fix skipped test

* added issue for skipped test
2023-10-23 18:08:33 +03:00
Florin Barbu
80a33b9c2b
chore: allure test reporting (#1668)
* allure test reporting

* make reports only for the main job

* fail a test for demo

* fail a test for demo

* revert

* fail a test for demo

* revert

* deploy to allure-jswaku

* use PAT

* add pat as secret

* check if pat exists

* debug secrets issue

* remove debug steps

* debug1

* debug2

* debug3

* debug4

* set reports for all tests

* fix after ci run

* self review

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2023-10-23 17:53:56 +03:00
Danish Arora
0f7d63ef93
feat: Logger with log levels (#1672)
* setup a custom Logger with log level support

* refactor codebase for to use new Logger with log levels

* disallow usage of `debug` directly / only allow usage in/through custom Logger

* remove `debug` from logger
2023-10-20 16:36:47 +05:30
Florin Barbu
1150ddcd02
fix: remote peer rejected (#1645)
* update empty payload light push tests response

* bump up nwaku version

* remove only

* remove diff between gowaku and nwaku
2023-10-20 10:21:15 +03:00
Danish Arora
9fe04d85f3
chore: use pubsubTopic/pubsubTopics across the codebase (#1653)
* chore: change all references of pubSubTopic to pubsubTopic

* change references of pubSubTopics to pubsubTopics

* flag words in cspell
2023-10-16 12:52:32 +05:30
Florin Barbu
b96c3bd3e1
chore: disable paralles runs locally (#1662)
* disable paralles runs locally

* add logs

* add readme info about local execution
2023-10-13 17:32:28 +03:00
Florin Barbu
347cbfa08a
chore: run tests in parallel (#1655)
* run tests in parallel

* small fixes

* small fixes

* fix setup of nodes

* fix relay tests

* fix Static Sharding: Running Nodes tests

* try with 5 threads

* try with 6 threads

* use startWithRetries as default start

* revert to 6

* set 10 jobs

* revert to back to 6

* add CI info in readme
2023-10-13 12:36:43 +03:00
fbarbu15
aa93a441d5
use loadash instead of custom function 2023-10-10 09:15:36 +03:00
fbarbu15
c9389e3d07
improve teardown nodes 2023-10-10 09:03:44 +03:00