Commit Graph

2955 Commits

Author SHA1 Message Date
Danish Arora f0eb244804
chore: refactor sub dir paths 2024-08-20 19:07:07 +05:30
Danish Arora 1de1fd420e
feat(filter): decouple reliability logic to a separate class 2024-08-20 18:49:55 +05:30
Danish Arora 409642d7d3
chore: upgrade libp2p and related packages (#2084)
* chore: upgrade libp2p and related packages

* fix: public key on peer id can now be undefined

* chore: upgrade remainder packages

* chore: fix tests

* chore: address comments

* chore: upgrade playwright and proto
2024-08-14 04:23:04 +05:30
Danish Arora 5ce36c8f18
feat!: deprecate named pubsub topics and use static/auto sharding (#2097)
* feat: deprecate named sharding & protocols adhere
simplify network config type, all protocols use pubsub topic internally

* chore: update tests

* tests: rm application info

* chore: use static sharding and auto sharding terminologies

* chore: update docs for network config

* chore: update interfaces

* tests: update tests error message

* chore: remove `ShardingParams` type and fix test
2024-08-13 05:23:20 +05:30
Danish Arora 86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Danish Arora fdd9dc44a4
feat(filter)!: new simpler filter API (#2092)
* chore: rename IReceiver subscribe

* feat!: new `subscribe() API that only takes in decoders and callback

* chore: `to_async_iterator` uses new function name

* chore: make `createSubscription` private, and shorten error handling

* chore: update subscribe return type

* tests: use new API

* fix: tests
2024-08-05 15:52:58 +05:30
Danish Arora fea4f2577b
chore(message-hash): use `timestamp` in nanoseconds instead of milliseconds (#2094)
* chore: convert timestamp from miliseconds to nanoseconds

* chore: test against hash accounting for nanoseconds instead of milliseconds
the hash used was calculated using timestamp in milliseconds, instead of nanoseconds (not part of the RFC test vectors)Y
2024-08-01 15:19:35 +05:30
Danish Arora 33a9172bbb
chore: upgrade nwaku to 0.31 (#2091)
* chore: upgrade nwaku to 0.31

* tests: fix tests by removing reliance on named sharding
2024-08-01 14:33:42 +05:30
Danish Arora d464af3645
feat: node and protocols health (#2080)
* feat: introduce HealthManager

* feat: make health accessible on Waku object

* feat: update health from protocols

* chore: add access modifiers to healthmanager

* feat: use a HealthManager singleton

* chore: add tests for Filter, LightPush and Store

* feat: add overall node health

* chore: update protocol health to consider Store protocol

* chore: setup generic test utils instead of using filter utils

* tests: add a health status matrix check from 0-3

* chore: increase timeout for failing tests in CI
tests pass locally without an increased timeout, but fail in CI

* chore: move name inference to HealthManager

* tests: abstract away node creation and teardown utils

* fix: import
2024-07-27 18:27:54 +05:30
Danish Arora defe41bb9a
feat(peer-exchange): support continuous peer information updates (#2088)
* feat(peer-exchange): update peer info if new is found

* chore: move diff checking logic to a private function

* chore: add tests

* chore: increase verbosity for mulltiaddr

* chore: use merge for metadata nad patch for multiaddrs

* chore: use peerId from peerInfo

* chore: remove unused import
2024-07-26 17:14:55 +05:30
gabrielmer 08fc2d133a
chore: changing default pubsub topic to its static sharding version (#2083)
* changing default pubsub topic to its static sharding version

* keeping RFC's Waku Message test vectors

* reverting change in changelog

* setting pubsub topic when creating nwaku node

* adding shardInfo to runMultipleNodes call

* adding shardInfo to runMultipleNodes call in lightpush tests

* add pubsub topics to nwaku.start

* get rid of it.only that remained

* fixing compliance tests

* setting clusterId to 0

* removing unnecessary fix

* adding shardInfo when creating nodes

* fixing wait for remote peer tests

* fixing peer exchange test

* refactor

* removing unnecessary variable

* feat: create default shard info, update tests (#2085)

* feat: create default shard info, update tests

* add link

* fix tests

* remoe only

* up tests

* up test

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2024-07-26 00:21:52 +02:00
Sasha 04bd518210
chore: remove content_topic specific API (#2081) 2024-07-23 17:25:36 +02:00
Danish Arora a739ada33a
chore(peer-exchange): use an event listener to gauge if the service is mounted (#2071)
* chore: use service mounted event listener

* chore: update event name

* chore: change event name to isStarted

* chore: address comments
2024-07-22 18:01:15 +05:30
Sasha 2b02f829c2
chore: throw if more than one network config is passed (#2056)
* chore: throw if more than one network config is passed

* up

* update init

* up message

* improve project structure

* address nit
2024-07-19 19:35:00 +02:00
Danish Arora 169a09d552
chore: enforce access modifiers (#2068)
* feat: introduce eslint flag

* chore: update logger

* chore: update enr

* chore: update core

* chore: update sdk

* chore: update relay

* chore: update discovery

* chore: update message-encryption

* chore: update tests

* chore: fix modifiers

* chore(tests): fix access modifiers

* chore: fix rebase
2024-07-19 15:58:17 +05:30
Danish Arora 9b0f1e855a
feat: validate messages for individual filter nodes & perform renewals (#2057)
* feat: validate messages for individual filter nodes & perform renewals

* chore: fix spell check

* chore: use a max threshold before peer renewal

* chore: switch from a validation cycle timer to adhoc validation

* chore: add test

* fix: test

* chore: address comments

* fix: renewal without a new peer available

* chore: validating messages should be non-blocking

* chore: minor improvements

* chore: rm only

* chore: fix test
2024-07-18 00:41:21 +05:30
Sasha 00635b7afe
feat: fix peer renewal, change Filter keep alive (#2065)
* move util, create stream manager folder

* change default keep alive, improve peer renewal

* address nit
2024-07-16 18:35:24 +02:00
Sasha 09a81302c1
chore: bump @waku peer dependencies (#2062) 2024-07-16 12:00:36 +02:00
dependabot[bot] c0c2907131
chore(deps)(deps): bump fast-check from 3.17.1 to 3.19.0 (#2060)
Bumps [fast-check](https://github.com/dubzzz/fast-check/tree/HEAD/packages/fast-check) from 3.17.1 to 3.19.0.
- [Release notes](https://github.com/dubzzz/fast-check/releases)
- [Changelog](https://github.com/dubzzz/fast-check/blob/main/packages/fast-check/CHANGELOG.md)
- [Commits](https://github.com/dubzzz/fast-check/commits/v3.19.0/packages/fast-check)

---
updated-dependencies:
- dependency-name: fast-check
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 23:51:25 +05:30
dependabot[bot] 14f84017be
chore(deps)(deps): bump sinon from 17.0.1 to 18.0.0 (#2058)
Bumps [sinon](https://github.com/sinonjs/sinon) from 17.0.1 to 18.0.0.
- [Release notes](https://github.com/sinonjs/sinon/releases)
- [Changelog](https://github.com/sinonjs/sinon/blob/main/docs/changelog.md)
- [Commits](https://github.com/sinonjs/sinon/compare/v17.0.1...v18.0.0)

---
updated-dependencies:
- dependency-name: sinon
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-10 21:34:26 +05:30
Sasha ccdf84e593
chore: release master (#2004)
reliability improvements
2024-07-10 17:25:22 +02:00
Danish Arora 318667e442
feat(filter): peer/subscription renewal with recurring Filter pings (#2052)
* chore: renewPeer() returns the new found peer

* feat: ping & peer renewal

* chore: add tests

* fix: tests

* chore: remove only

* chore: remove comments

* chore(tests): decrease timeout

* chore: add array index validation

* chore: remove only

* chore: move defaults into a separate variable

* chore: update lightpush with new API

* chore: include peer renewals within `ping` instead of `interval`

* chore: update tests

* chore: add new test

* chore: address comments
2024-07-10 15:34:16 +05:30
dependabot[bot] 68590f0a3d
chore(deps)(deps-dev): bump serve from 14.2.1 to 14.2.3 (#2041)
Bumps [serve](https://github.com/vercel/serve) from 14.2.1 to 14.2.3.
- [Release notes](https://github.com/vercel/serve/releases)
- [Commits](https://github.com/vercel/serve/compare/14.2.1...14.2.3)

---
updated-dependencies:
- dependency-name: serve
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-10 02:48:19 +02:00
Danish Arora 4db508b962
feat(filter): use protocol peer management (#2047)
* feat: leverage protocol peer management

* chore: add test

* chore: address comments

* chore: add todo
2024-07-03 12:09:34 +05:30
Danish Arora 42126a6301
chore: upgrade nwaku to 0.29.0 (#2042) 2024-06-20 11:30:55 -04:00
Danish Arora 93e78c3b87
feat(lightpush): peer management for protocols (#2003)
* chore: make `dropConnection` to be a public function

* feat: peers are maintained for protocols
- passes `ConnectionManager` to ProtocolSDK to allow disconnecting from within protocol
- maintains `numPeersToUse` for each protocol within BaseProtocolSDK

* fix: pass options to protocols

* chore: update interfaces to allow public access

* chore: improve logging on protocol

* fix: renew peer upon failure

* chore(tests): allow DefaultPubsubTopic

* feat(lightpush): write peer management tests

* chore: rename test

* feat: add lock to `maintainPeers()` to handle parallelisation of requests
fixes parallelisation of lightpush.send() requests

* fix: concurrent lightpush requests

* fix: test & improve peers fetching

* chore: use getter

* address comments

* chore: smaller improvements

* feat: attempt to improve time for first lightpush.send()

* chore: use `window.interval` for type-safety

* chore: remove delays

* feat: add autoRetry

* feat: `forceUseAllPeers` to wait for all connected peers to be resoled
2024-06-19 01:52:16 -04:00
dependabot[bot] 69f9045bed
chore(deps)(deps-dev): bump expo from 50.0.14 to 51.0.14 (#2038)
Bumps [expo](https://github.com/expo/expo/tree/HEAD/packages/expo) from 50.0.14 to 51.0.14.
- [Changelog](https://github.com/expo/expo/blob/main/packages/expo/CHANGELOG.md)
- [Commits](https://github.com/expo/expo/commits/HEAD/packages/expo)

---
updated-dependencies:
- dependency-name: expo
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-18 16:12:48 -04:00
Danish Arora 984fb94b5b
fix: failing `node_optional` check (#2025)
* chore: remove predefine nodes list

* remove import

* chore: update & fix test

* chore: remove console logs
2024-06-18 13:43:22 -04:00
Danish Arora 16e9116c7c
fix: bootstrapping with default pubsub topic (#2031)
* fix: bootstrapping into default pubsub topic

* chore: update tests

* chore: update TODO with GH issue

* chore: rename fleets

* feat: use TWN fleets, or wakuv2.prod if DefaultPubsubTopic

* chore: update imports for enrtree
2024-06-06 12:55:57 -04:00
fbarbu15 c5302fd0c8
remove try catch no longer needed after 1790 got fixed (#2026) 2024-05-30 21:29:16 +05:30
gabrielmer 2311a595b2
fix: increasing maxInboundStreams for lightpush from 32 to 100 (#2021)
* increasing maxInboundStreams for lightpush from 32 to 100

* allowing test to fail
2024-05-30 13:58:50 +05:30
Danish Arora e49e7289ae
chore: improve StreamManager (#1994)
* chore: improve stream manager

* chore: some cleaning

* chore: update API

* chore: rename `getConnectionStatus` to `isConnected`

* chore: use throw/catch

* chore: remove await

* remove redundant function and rename

* chore: increase test timeout
2024-05-14 16:31:38 +05:30
Arseniy Klempner 6f188ef379
fix: add .js to base protocol import in filter sdk (#2009) 2024-05-14 13:06:10 +05:30
Arseniy Klempner f78495a5a2
Merge pull request #1997 from waku-org/feat/docker-sepolia-rpc
feat: add rpc url to nwaku, persist rln tree in docker and ci
2024-05-13 09:41:04 -07:00
Arseniy Klempner aad819bb1f
feat: add rpc url to nwaku, persist rln tree in docker and ci 2024-05-13 07:42:08 -07:00
Danish Arora 4eb06c64eb
feat(filter)!: return error codes instead of throwing errors (#1971)
* move protocol result type to interfaces

* chore: update type names for verbosity

* feat(filter-core): convert error throws to return types

* chore: update types & imports

* update Filter API

* chore: update createSubscription

* chore: update imports & rename

* chore: update all tests

* chore: resolve conflicts & merge (2/n)

* chore: resolve conflicts & merge (3/n)

* chore: resolve conflicts & merge (4/n)

* chore: resolve conflicts & merge (5/n)

* chore: resolve conflicts & merge (6/n)

* chore: use idiomatic approach

* chore: fix tests

* chore: address comments

* chore: fix test

* rm: only
2024-05-09 16:51:08 +05:30
Arseniy Klempner 5df41b0adf
fix: add shard arg to nwaku in peer management tests (#2005) 2024-05-09 10:25:35 +02:00
Arseniy Klempner 21aa22f320
Merge pull request #1989 from waku-org/fix/transport-filter-wss
fix: filter for wss in libp2p websocket transport
2024-05-06 16:45:57 -07:00
Arseniy Klempner 682cc66232
fix: filter for wss in libp2p websocket transport 2024-04-30 20:18:11 -07:00
Sasha 3d92f194a7
chore: release master (#1957) 2024-04-30 17:50:00 +02:00
Sasha e5293356d8
feat: make message hash compatible with decoded message (#1993)
* feat: make message hash compatible with decoded message

* add util fn

* fix test

* up hashes

* up
2024-04-30 15:37:10 +02:00
Sasha 5b03709dfe
feat: lift contentTopics and make shardInfo mandatory for createLight… (#1959)
* feat: lift contentTopics and make shardInfo mandatory for createLightNode

* add default shard info and wanring

* fix

* fix lint

* fix

* fix types

* remove breaking changes
2024-04-30 01:47:45 +02:00
Arseniy Klempner fb34b7262a
fix: only override ping metadata in peer store (#1984)
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2024-04-30 00:39:02 +02:00
Sasha 1a6bc4f8ce
feat: add keep alive to Filter (#1970)
* fix: use pubsubTopic from current ones if not set

* fix: improve type on dial method

* enforce same pubusb on filter.subscribe, make content topic to pubsub mapping default for decoder / encoder

* fix mapping problem

* update tests

* add error handling

* fix typo

* up lock

* rm lock

* up lock

* remove only

* feat: implement keep alive for filter subscription

* remove

* address comments
2024-04-29 23:31:09 +02:00
Arseniy Klempner bc98b4fb0d
Merge pull request #1987 from waku-org/feat/max-inbound-ping
feat: add libp2p option for max ping connections
2024-04-29 08:58:09 -07:00
Arseniy Klempner fa523b78af
feat: add libp2p option for max ping connections 2024-04-29 08:21:37 -07:00
Danish Arora 73d4f19746
fix(message-hash): account for `timestamp` (#1986)
* fix: message-hash algo + tests

* rm: only

* chore: move numberToBytes to utils package

* chore: shorten implementation
2024-04-29 15:35:23 +05:30
gabrielmer 1562f0fd6f
debug: fixing peer exchange tests (#1990)
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2024-04-29 11:22:36 +02:00
Sasha f3627c46a4
feat!: use ShardingParams on subscriptions, make Decoder/Encoder auto sharding friendly by default (#1958)
* fix: use pubsubTopic from current ones if not set

* fix: improve type on dial method

* enforce same pubusb on filter.subscribe, make content topic to pubsub mapping default for decoder / encoder

* fix mapping problem

* update tests

* add error handling

* fix typo

* up lock

* rm lock

* up lock

* remove only

* fix content topic

* fix ephemeral test

* fix filter unsubscribe test

* up utils

* fix subscribe test

* up interfaces and filter api

* remove only

* up ping test

* fix subscribe test

* fix push test

* fix lightPush

* fix multiple pubsub

* remove only, fix subscribe filter test

* remove only

* fix cluster ID selection and named sharding subscription test

* fix unsubscribe test

* fix light push test

* fix light push test

* fix push test

* fix relay publish

* create runNode and fix relay tests

* generalize runNodes, fix some tests

* fix store tests

* fix toAsyncIterator tests

* remove only

* fix lightPush

* use generics

* try fix test

* run failing tests

* remove only

* address failed tests, remove DefaultPubsubTopic dependency in some tests
2024-04-28 11:15:17 +02:00
fbarbu15 86249dfe29
fix: unskip shard 0 test fixed by 1874 (#1978) 2024-04-25 15:22:16 +05:30