Commit Graph

2983 Commits

Author SHA1 Message Date
Sasha c93b58ed8c
up lock 2024-10-19 17:02:38 +02:00
Sasha c591e6d294
chore: release master (#2135) 2024-10-17 14:06:38 +02:00
Sasha ded994f8ec
fix: attempt to fix some of the Filter issues (#2183)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* turn off check for missing messages

* fix recurring ping

* add useful logs

* skip tests

* remove comment

* feat: check filter subscriptions against lightPush (#2185)
2024-10-17 01:01:21 +02:00
Sasha 4049123f14
feat!: lighten retry logic for LightPush (#2182)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* skip tests
2024-10-17 00:49:24 +02:00
Danish Arora 19a5d29aa7
chore: change filter ping interval to 1 min (#2184) 2024-10-16 15:30:56 +05:30
Danish Arora b2efce5ec2
feat(filter): enhancing protocol peer management with mutex locks (#2137)
* chore: improvements

* chore: add logs for subscription maintenance

* chore: update logging

* chore: trimming down BaseProtocolCore

* chore: track peers in a hashmap instead of array

* chore: peer mgmt responds to conenction/disconnection and improve logging

* feat: add mutex locks to tackle race conditions over shared state

* fix: build

* chore: some mutex lock-release improvements

* feat: peer manager

* chore: rm tests for remove internal util

* chore: update HealthManager updates

* chore: update tests

* rm: only

* fix: hasPeers management

* chore: add modularity to getting connected peers

* chore: improve logs & add debug

* chore: renewal doesnt disconnect, only removes

* chore: await for sequential operations

* chore: add TODO

* chore: minor improvements

* chore: fix rebase

* chore: update playright

* chore: remove additional arg

* chore: update interafce

* feat(peer-manager): unit tests

* chore: improve hasPeers()

* chore: update lockfile

* feat: Filter reacts to peer:disconnect event, add tests

* chore: fix lock

* chore: update playright

* chore: update protocol health for lightpush

* chore: remove .only

* chore: address comments and improvements

* fix: tsconfig
2024-10-10 23:47:12 +02:00
Sasha 75fcca4cd9
feat: replace `waitForRemotePeers()` with `waku.waitForPeer()` method (#2161)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* add I prefix to Waku interface

* implement waku.connect method

* add docs to IWaku interface

* remove export and usage of waitForRemotePeer

* move wait for remote peer related to Realy out of @waku/sdk

* change tests to use new API

* fix linting

* update size limit

* rename .connect to .waitForPeer

* export waitForRemotePeer and mark as deprecated

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer (#2163)

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer

* add waitForRemote UTs

* remove junk

* feat: expose peerId and protocols from WakuNode (#2166)

* chore: expose peerId and protocols from WakuNode

* remove unused method

* move to private method

* rename to waitForPeers

* up test
2024-10-09 00:43:34 +02:00
Sasha d37e0245cf
feat: confirm metadata and protocols needed in waitForRemotePeer (#2160)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* imrpove iteration for existing connections

* address protocol adverisement in CI

* add protocols needed

* add missing protocols

* make lightpush and filter default for tests

* up
2024-10-05 01:49:54 +02:00
Sasha 2be0e81a0a
chore: rename IProtocolSDK interfaces to IProtocol naming convention (#2159)
* chore: rename IProtocolSDK interfaces to IProtocol
2024-10-04 13:53:54 +02:00
Sasha 1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Sasha b93134a517
fix: peer renewal connection drop & stream management (#2145)
* fix: peer renewal connection drop

* fix stream manager

* fix over iteration during stream creation

* remove timeout and use only open peers

* add logs

* refactor code, add tests

* debug test

* up debug

* remove debug, supress check for timestamps

* remove only

* add more debug

* remove debug

* remove check for timestamps
2024-10-01 12:54:55 +02:00
chair 3e821591c9
Merge pull request #2142 from waku-org/chair28980-update-codeowners-20240917
chore: Update CODEOWNERS to reference correct team
2024-09-17 23:42:08 -07:00
chair 1ed9ca5b33
Update CODEOWNERS to reference correct team 2024-09-17 21:57:54 -07:00
Danish Arora 7a6247cb70
feat(lightpush): introduce ReliabilityMonitor and allow `send` retries (#2130)
* chore: restructure reliabiltiy monitors

* feat: setup sender monitor

* chore: update tests

* chore: minor fixes

* chore: comment for doc
2024-09-17 11:34:59 +05:30
Danish Arora 7ad1d321ca
feat(filter): reliability monitor as a separate class to handle reliability logic (#2117)
* chore: move SubscriptionManager to a separate file

* feat: introduce ReliabilityMonitor

* fix: peer data updates

* fix: logical error when returning includesMessage

* chore: handle edge case

* chore: move ping failures handling inside monitor

* chore: move renewal logic to monitor

* chore: improve structuring

* chore: update logger

* chore: readd connectionListener() logic from merge

* chore: minor fixes

* chore: improve

* chore: setup destruction of ReliabilityMonitors

* fix: condition for ping failure

* fix: setters

* chore: handle race condition & fix test
2024-09-13 14:57:29 +05:30
Danish Arora 9d9a696024
chore: mandate `.js` extensions for imports (#2128)
* feat: add eslint rule

* chore: update codebase import extensions
2024-09-13 14:18:29 +05:30
Sasha a2c5dba32d
chore: release master (#2122) 2024-09-06 01:42:16 +02:00
Sasha 0263cb80c5
fix: improve node bootstrapping (#2121)
* fix: improve node bootstrapping

* implement dns discovery on failed dials

* update const to match RFC

* up
2024-09-06 01:10:54 +02:00
Sasha f4b6bb04b3
fix: temporarily remove peer cross dependencies (#2123)
* chore: temporarily remove peer cross dependencies

* up lock

* add missing dep
2024-09-06 00:44:59 +02:00
Sasha 5d3cc5fd45
fix: filter missing messages (#2119)
* bug: fix filter missing messages

* fix keep alive

* fix const
2024-09-03 00:59:21 +02:00
Sasha 5cfe9327b6
chore: remove relay from sdk package (#2040)
* remove full node

* remove relay from sdk

* comment node counter check

* try using logline

* up comment
2024-08-31 15:18:51 +02:00
Sasha f9361bf774
chore: release master (#2066) 2024-08-29 11:49:04 +02:00
Sasha a022433851
feat(lightpush)!: return new error messages (#2115)
* feat: return new error messages

* fix test, remove unused

* up

* up

* rollback

* up test
2024-08-29 11:20:19 +02:00
Sasha eadb85ab83
feat: offline state recovery for Filter subscription (#2049)
* up

* fix window reference

* add tests

* up

* add e2e renew test

* address comments

* remove unused

* add test

* try

* remove only

* up test

* up

* remove only

* add tmp logs, use before/after hooks

* up

* fix check

* remove only

* fix test

* up
2024-08-28 18:00:18 +02:00
Sasha 71384dfdfd
chore: change amount of used peers to 2 (#2111)
* chore: change amount of used peers to 2

* up

* up

* up tests
2024-08-27 21:53:16 +02:00
chair f0a5c6f056
chore: update add-action-project.yml (#2114) 2024-08-27 11:11:35 +02:00
Danish Arora 69b6612ec9
chore: upgrade node to LTS 20 (#2110)
* chore: upgrade nodejs to new LTS 20

* chore: update other references
2024-08-23 15:57:20 +05:30
Sasha 8f56d90cf1
fix: import of base_protocol and networkConfig type guard (#2109)
* bug: fix import

* fix type guard
2024-08-23 11:44:55 +02:00
Sasha c33844eddf
chore: update husky to new api (#2102) 2024-08-22 18:01:29 +02:00
Sasha 2beb359cef
chore: enabile back filterAll if set by consumer (#2103) 2024-08-22 15:55:29 +02:00
Danish Arora 409642d7d3
chore: upgrade libp2p and related packages (#2084)
* chore: upgrade libp2p and related packages

* fix: public key on peer id can now be undefined

* chore: upgrade remainder packages

* chore: fix tests

* chore: address comments

* chore: upgrade playwright and proto
2024-08-14 04:23:04 +05:30
Danish Arora 5ce36c8f18
feat!: deprecate named pubsub topics and use static/auto sharding (#2097)
* feat: deprecate named sharding & protocols adhere
simplify network config type, all protocols use pubsub topic internally

* chore: update tests

* tests: rm application info

* chore: use static sharding and auto sharding terminologies

* chore: update docs for network config

* chore: update interfaces

* tests: update tests error message

* chore: remove `ShardingParams` type and fix test
2024-08-13 05:23:20 +05:30
Danish Arora 86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Danish Arora fdd9dc44a4
feat(filter)!: new simpler filter API (#2092)
* chore: rename IReceiver subscribe

* feat!: new `subscribe() API that only takes in decoders and callback

* chore: `to_async_iterator` uses new function name

* chore: make `createSubscription` private, and shorten error handling

* chore: update subscribe return type

* tests: use new API

* fix: tests
2024-08-05 15:52:58 +05:30
Danish Arora fea4f2577b
chore(message-hash): use `timestamp` in nanoseconds instead of milliseconds (#2094)
* chore: convert timestamp from miliseconds to nanoseconds

* chore: test against hash accounting for nanoseconds instead of milliseconds
the hash used was calculated using timestamp in milliseconds, instead of nanoseconds (not part of the RFC test vectors)Y
2024-08-01 15:19:35 +05:30
Danish Arora 33a9172bbb
chore: upgrade nwaku to 0.31 (#2091)
* chore: upgrade nwaku to 0.31

* tests: fix tests by removing reliance on named sharding
2024-08-01 14:33:42 +05:30
Danish Arora d464af3645
feat: node and protocols health (#2080)
* feat: introduce HealthManager

* feat: make health accessible on Waku object

* feat: update health from protocols

* chore: add access modifiers to healthmanager

* feat: use a HealthManager singleton

* chore: add tests for Filter, LightPush and Store

* feat: add overall node health

* chore: update protocol health to consider Store protocol

* chore: setup generic test utils instead of using filter utils

* tests: add a health status matrix check from 0-3

* chore: increase timeout for failing tests in CI
tests pass locally without an increased timeout, but fail in CI

* chore: move name inference to HealthManager

* tests: abstract away node creation and teardown utils

* fix: import
2024-07-27 18:27:54 +05:30
Danish Arora defe41bb9a
feat(peer-exchange): support continuous peer information updates (#2088)
* feat(peer-exchange): update peer info if new is found

* chore: move diff checking logic to a private function

* chore: add tests

* chore: increase verbosity for mulltiaddr

* chore: use merge for metadata nad patch for multiaddrs

* chore: use peerId from peerInfo

* chore: remove unused import
2024-07-26 17:14:55 +05:30
gabrielmer 08fc2d133a
chore: changing default pubsub topic to its static sharding version (#2083)
* changing default pubsub topic to its static sharding version

* keeping RFC's Waku Message test vectors

* reverting change in changelog

* setting pubsub topic when creating nwaku node

* adding shardInfo to runMultipleNodes call

* adding shardInfo to runMultipleNodes call in lightpush tests

* add pubsub topics to nwaku.start

* get rid of it.only that remained

* fixing compliance tests

* setting clusterId to 0

* removing unnecessary fix

* adding shardInfo when creating nodes

* fixing wait for remote peer tests

* fixing peer exchange test

* refactor

* removing unnecessary variable

* feat: create default shard info, update tests (#2085)

* feat: create default shard info, update tests

* add link

* fix tests

* remoe only

* up tests

* up test

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2024-07-26 00:21:52 +02:00
Sasha 04bd518210
chore: remove content_topic specific API (#2081) 2024-07-23 17:25:36 +02:00
Danish Arora a739ada33a
chore(peer-exchange): use an event listener to gauge if the service is mounted (#2071)
* chore: use service mounted event listener

* chore: update event name

* chore: change event name to isStarted

* chore: address comments
2024-07-22 18:01:15 +05:30
Sasha 2b02f829c2
chore: throw if more than one network config is passed (#2056)
* chore: throw if more than one network config is passed

* up

* update init

* up message

* improve project structure

* address nit
2024-07-19 19:35:00 +02:00
Danish Arora 169a09d552
chore: enforce access modifiers (#2068)
* feat: introduce eslint flag

* chore: update logger

* chore: update enr

* chore: update core

* chore: update sdk

* chore: update relay

* chore: update discovery

* chore: update message-encryption

* chore: update tests

* chore: fix modifiers

* chore(tests): fix access modifiers

* chore: fix rebase
2024-07-19 15:58:17 +05:30
Danish Arora 9b0f1e855a
feat: validate messages for individual filter nodes & perform renewals (#2057)
* feat: validate messages for individual filter nodes & perform renewals

* chore: fix spell check

* chore: use a max threshold before peer renewal

* chore: switch from a validation cycle timer to adhoc validation

* chore: add test

* fix: test

* chore: address comments

* fix: renewal without a new peer available

* chore: validating messages should be non-blocking

* chore: minor improvements

* chore: rm only

* chore: fix test
2024-07-18 00:41:21 +05:30
Sasha 00635b7afe
feat: fix peer renewal, change Filter keep alive (#2065)
* move util, create stream manager folder

* change default keep alive, improve peer renewal

* address nit
2024-07-16 18:35:24 +02:00
Sasha 09a81302c1
chore: bump @waku peer dependencies (#2062) 2024-07-16 12:00:36 +02:00
dependabot[bot] c0c2907131
chore(deps)(deps): bump fast-check from 3.17.1 to 3.19.0 (#2060)
Bumps [fast-check](https://github.com/dubzzz/fast-check/tree/HEAD/packages/fast-check) from 3.17.1 to 3.19.0.
- [Release notes](https://github.com/dubzzz/fast-check/releases)
- [Changelog](https://github.com/dubzzz/fast-check/blob/main/packages/fast-check/CHANGELOG.md)
- [Commits](https://github.com/dubzzz/fast-check/commits/v3.19.0/packages/fast-check)

---
updated-dependencies:
- dependency-name: fast-check
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 23:51:25 +05:30
dependabot[bot] 14f84017be
chore(deps)(deps): bump sinon from 17.0.1 to 18.0.0 (#2058)
Bumps [sinon](https://github.com/sinonjs/sinon) from 17.0.1 to 18.0.0.
- [Release notes](https://github.com/sinonjs/sinon/releases)
- [Changelog](https://github.com/sinonjs/sinon/blob/main/docs/changelog.md)
- [Commits](https://github.com/sinonjs/sinon/compare/v17.0.1...v18.0.0)

---
updated-dependencies:
- dependency-name: sinon
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-10 21:34:26 +05:30
Sasha ccdf84e593
chore: release master (#2004)
reliability improvements
2024-07-10 17:25:22 +02:00
Danish Arora 318667e442
feat(filter): peer/subscription renewal with recurring Filter pings (#2052)
* chore: renewPeer() returns the new found peer

* feat: ping & peer renewal

* chore: add tests

* fix: tests

* chore: remove only

* chore: remove comments

* chore(tests): decrease timeout

* chore: add array index validation

* chore: remove only

* chore: move defaults into a separate variable

* chore: update lightpush with new API

* chore: include peer renewals within `ping` instead of `interval`

* chore: update tests

* chore: add new test

* chore: address comments
2024-07-10 15:34:16 +05:30