3004 Commits

Author SHA1 Message Date
Danish Arora
0a0a92bccb
feat: @waku/rln (#2244)
* chore: setup rln as a new package

* chore: migrate src

* fix: wasm loading, tests, config

* chore: fix Karma CI

* fix: bundler

* chore: copy dist resources

* chore(rln): enable all tests

* chore: increase karma timeouts
2025-02-11 15:28:00 +05:30
Danish Arora
becb46f3a5
chore(sds): use allure reporting (#2255) 2025-02-11 00:56:00 +05:30
Arseniy Klempner
053e4901e7
Merge pull request #2253 from waku-org/feat/sds-bigint-nimhashn
feat(sds): migrate bloomfilter to bigint and import hashn function from nim
2025-02-07 11:59:27 -08:00
Arseniy Klempner
be93e4b71f
feat(sds): migrate bloomfilter to bigint and import hashn function from nim
Uses an array of bigint to store sufficient bits in bloom filter.
Updates all arithmetic to explicitly cast to bigint where necessary.
Makes the hashn function for bloomfilter a parameter.
Adds an implementation of hashn generated using nim compiler.
Adds tests.
2025-02-05 17:27:41 -08:00
Sasha
3136f3a704
feat: add HealthIndicator with simplified logic and testing (#2251)
* implement HealthIndicator

* up libp2p interface version

* up lock

* remove unused tests

* expose HealthIndicator from Waku

* update test, add start and stop

* fix error handling
2025-02-05 13:24:50 +01:00
Sasha
fc93fae873
feat: move Peer to PeerId (#2246)
* feat: move Peer to PeerId

* up tests

* update tests
2025-02-05 11:21:22 +01:00
fbarbu15
2a7f4b6e86
chore: add back allure (#2247) 2025-02-03 13:59:55 +05:30
Sasha
62f93dc842
feat: improve peer manager and re-integrate to light push (#2191)
* up lock

* make ConnectionManager use ctor

* reform connection manager configurations

* remove log param from peerManager

* make PeerManager use only ConnectionManager, move getPeers to ConnectionManager, remove not needed code

* remove allPeers and connectedPeers from BaseProtocolCore, update tests, add getPeers for IWaku

* use only one peerManager from Waku object

* remove IBaseProtocolSDK and merge with PeerManager

* re-implement peerManager, remove ProtocolUseOptions

* remove not needed test, up lock

* update deps and lock

* remove old test for peerManager, fix check and spell

* rename to getConnectedPeers

* feat: improve filter subscriptions (#2193)

* add message cache to Filter

* remove WakuOptions and use only ProtocolCreateOptions

* move subscribe options to createLightNode Fitler protocol options

* rename SubscriptionManager to Subscription

* rename to CreateNodeOptions

* add warning

* feat: introduce subscription manager (#2202)

* feat: inroduce subscription manager

* fix: make pipeline succeed (#2238)

* fix test

* use hardcoded value

* update playwright

* fix test:browser

* up lock

* make peer retrieval probabilistic

* add comments

* up lightpush tests

* add tests for peer_manager, improve folder structure

* create named files for protocols

* create named files, simplify project structure

* remove only
2025-01-31 00:16:00 +01:00
Ivan FB
d706fe146c
replace pubsub topic param with clusterid plus shard (#2239)
We are in the process of deprecating pubsub-topic param from nim-waku.
The pubsub-topic shouldn't be used to start nim-waku and instead, a
combination of 1 cluster-id plus n shard should be given. This aims to
start using the new added param: shard.
2025-01-30 18:16:07 +05:30
Arseniy Klempner
6abd2d18a1
Merge pull request #2233 from waku-org/feat/init-sds
feat(sds): create package for sds and add protobuf def
2025-01-29 11:48:29 -08:00
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00
Danish Arora
f2ce43c7d9
chore: remove log level to make logs readable in the CI (#2237) 2025-01-28 20:43:39 +05:30
Danish Arora
4153396014
feat(store): allow specifying node to use (#2192)
* feat: API allows using specific nodes for protocol

* chore: dial provided now

* chore: store uses the node

* chore: update API

* chore: use protocol-level class for getting peer

* chore: use ConnectionManager for dial ops

* chore: address comments

* chore: fix type error

* chore: Waku.dial() proxies through ConnectionManager.dialPeer

* chore: fix dial

* chore: add ts-doc for dialPeer()

* chore: remove log

* chore: reduce ts-doc for an internal function

* chore: address comments

* chore: return types from connmanager.dialpeer()

* chore: reduce diff by not introducing breaking changes
2025-01-28 17:57:49 +05:30
Danish Arora
a73dd4f083
chore: improve naming for test reports job (#2234) 2025-01-28 16:24:16 +05:30
Danish Arora
e7bb092fac
chore: upgrade upload-artifacts (#2221) 2025-01-28 15:44:07 +05:30
Danish Arora
f3aed7074b
chore: switch out of allure reporters (#2222)
* chore: switch out of allure

* chore: fix config

* fix: test file patterns
2025-01-27 21:14:34 +05:30
Sasha
4b28f250e3
chore: update issue templates (#2206)
* feat: update issue templates

* remove old issue template, add new pull request template
2025-01-16 14:57:14 +01:00
Sasha
f34fc4b244
fix: remove peer deps (#2200) 2025-01-16 14:27:46 +01:00
Sasha
b5d2c013a1
fix access to process variable (#2201) 2025-01-16 13:32:40 +01:00
Danish Arora
ef78e52c45
fix: CI browser/karma (#2204)
* fix: browser tests

* chore: remove redundant replacement
2025-01-16 12:52:08 +05:30
Sasha
88e33a90fd
fix: remove window reference and improve waitForRemotePeer (#2194)
* fix: remove window reference and improve waitForRemotePeer

* remove only

* up lock

* debug: try * as version

* return version

* up lock
2024-11-08 12:34:31 +07:00
Danish Arora
7c0ce7b2ec
chore: upgrade to libp2p v2 (#2143)
* chore: upgrade libp2p to v2 and related deps

* chore: fix ENR

* chore(core): remove CustomEvent polyfill import

* chore: `peer-id-factory` has been removed

* chore(discovery): fix local-cache & remove CustomEvent imports

* chore(sdk): update config

* chore(tests): update tests without peer-id-factory

* fix: spec tests

* chore: fix test

* chore: upgrade dataset-core

* chore: upgrade libp2p and stale references

* chore: upgrade playwright

* chore: rm console log

* fix: lock
2024-10-21 16:43:24 +05:30
Sasha
c591e6d294
chore: release master (#2135) sdk-v0.0.29 utils-v0.0.21 discovery-v0.0.6 core-v0.0.33 enr-v0.0.27 interfaces-v0.0.28 message-encryption-v0.0.31 message-hash-v0.1.17 relay-v0.0.16 2024-10-17 14:06:38 +02:00
Sasha
ded994f8ec
fix: attempt to fix some of the Filter issues (#2183)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* turn off check for missing messages

* fix recurring ping

* add useful logs

* skip tests

* remove comment

* feat: check filter subscriptions against lightPush (#2185)
2024-10-17 01:01:21 +02:00
Sasha
4049123f14
feat!: lighten retry logic for LightPush (#2182)
* feat: lighten retry logic for LightPush

* update tests

* remove base protocol sdk from light push, add unit tests for light push

* remove replaced test

* ensure numPeersToUse is respected

* skip tests
2024-10-17 00:49:24 +02:00
Danish Arora
19a5d29aa7
chore: change filter ping interval to 1 min (#2184) 2024-10-16 15:30:56 +05:30
Danish Arora
b2efce5ec2
feat(filter): enhancing protocol peer management with mutex locks (#2137)
* chore: improvements

* chore: add logs for subscription maintenance

* chore: update logging

* chore: trimming down BaseProtocolCore

* chore: track peers in a hashmap instead of array

* chore: peer mgmt responds to conenction/disconnection and improve logging

* feat: add mutex locks to tackle race conditions over shared state

* fix: build

* chore: some mutex lock-release improvements

* feat: peer manager

* chore: rm tests for remove internal util

* chore: update HealthManager updates

* chore: update tests

* rm: only

* fix: hasPeers management

* chore: add modularity to getting connected peers

* chore: improve logs & add debug

* chore: renewal doesnt disconnect, only removes

* chore: await for sequential operations

* chore: add TODO

* chore: minor improvements

* chore: fix rebase

* chore: update playright

* chore: remove additional arg

* chore: update interafce

* feat(peer-manager): unit tests

* chore: improve hasPeers()

* chore: update lockfile

* feat: Filter reacts to peer:disconnect event, add tests

* chore: fix lock

* chore: update playright

* chore: update protocol health for lightpush

* chore: remove .only

* chore: address comments and improvements

* fix: tsconfig
2024-10-10 23:47:12 +02:00
Sasha
75fcca4cd9
feat: replace waitForRemotePeers() with waku.waitForPeer() method (#2161)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* add I prefix to Waku interface

* implement waku.connect method

* add docs to IWaku interface

* remove export and usage of waitForRemotePeer

* move wait for remote peer related to Realy out of @waku/sdk

* change tests to use new API

* fix linting

* update size limit

* rename .connect to .waitForPeer

* export waitForRemotePeer and mark as deprecated

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer (#2163)

* feat: add mocha tests to @waku/sdk and cover waitForRemotePeer

* add waitForRemote UTs

* remove junk

* feat: expose peerId and protocols from WakuNode (#2166)

* chore: expose peerId and protocols from WakuNode

* remove unused method

* move to private method

* rename to waitForPeers

* up test
2024-10-09 00:43:34 +02:00
Sasha
d37e0245cf
feat: confirm metadata and protocols needed in waitForRemotePeer (#2160)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers

* feat: confirm metadata and protocols needed in waitForRemotePeer

* rely on passed protocols and fallback to mounted

* imrpove iteration for existing connections

* address protocol adverisement in CI

* add protocols needed

* add missing protocols

* make lightpush and filter default for tests

* up
2024-10-05 01:49:54 +02:00
Sasha
2be0e81a0a
chore: rename IProtocolSDK interfaces to IProtocol naming convention (#2159)
* chore: rename IProtocolSDK interfaces to IProtocol
2024-10-04 13:53:54 +02:00
Sasha
1d68526e72
feat(lightPush): improve peer usage and improve readability (#2155)
* fix comment of default number of peers

* export default number of peers from base protocol sdk

* rename to light_push, move class to separate file

* move waitForRemotePeer to sdk package

* add todo to move waitForGossipSubPeerInMesh into @waku/relay

* clean up waitForRemotePeer, split metadata await from event and optimise, decouple from protocol implementations

* simplify and rename ILightPush interface

* use only connected peers in light push based on connections instead of peer renewal mechanism

* improve readability of result processing in light push

* fix check & update tests

* address tests, add new test cases, fix racing condition in StreamManager

* use libp2p.getPeers
2024-10-04 10:50:58 +02:00
Sasha
b93134a517
fix: peer renewal connection drop & stream management (#2145)
* fix: peer renewal connection drop

* fix stream manager

* fix over iteration during stream creation

* remove timeout and use only open peers

* add logs

* refactor code, add tests

* debug test

* up debug

* remove debug, supress check for timestamps

* remove only

* add more debug

* remove debug

* remove check for timestamps
2024-10-01 12:54:55 +02:00
chair
3e821591c9
Merge pull request #2142 from waku-org/chair28980-update-codeowners-20240917
chore: Update CODEOWNERS to reference correct team
2024-09-17 23:42:08 -07:00
chair
1ed9ca5b33
Update CODEOWNERS to reference correct team 2024-09-17 21:57:54 -07:00
Danish Arora
7a6247cb70
feat(lightpush): introduce ReliabilityMonitor and allow send retries (#2130)
* chore: restructure reliabiltiy monitors

* feat: setup sender monitor

* chore: update tests

* chore: minor fixes

* chore: comment for doc
2024-09-17 11:34:59 +05:30
Danish Arora
7ad1d321ca
feat(filter): reliability monitor as a separate class to handle reliability logic (#2117)
* chore: move SubscriptionManager to a separate file

* feat: introduce ReliabilityMonitor

* fix: peer data updates

* fix: logical error when returning includesMessage

* chore: handle edge case

* chore: move ping failures handling inside monitor

* chore: move renewal logic to monitor

* chore: improve structuring

* chore: update logger

* chore: readd connectionListener() logic from merge

* chore: minor fixes

* chore: improve

* chore: setup destruction of ReliabilityMonitors

* fix: condition for ping failure

* fix: setters

* chore: handle race condition & fix test
2024-09-13 14:57:29 +05:30
Danish Arora
9d9a696024
chore: mandate .js extensions for imports (#2128)
* feat: add eslint rule

* chore: update codebase import extensions
2024-09-13 14:18:29 +05:30
Sasha
a2c5dba32d
chore: release master (#2122) message-encryption-v0.0.30 relay-v0.0.15 sdk-v0.0.28 interfaces-v0.0.27 enr-v0.0.26 message-hash-v0.1.16 core-v0.0.32 discovery-v0.0.5 utils-v0.0.20 2024-09-06 01:42:16 +02:00
Sasha
0263cb80c5
fix: improve node bootstrapping (#2121)
* fix: improve node bootstrapping

* implement dns discovery on failed dials

* update const to match RFC

* up
2024-09-06 01:10:54 +02:00
Sasha
f4b6bb04b3
fix: temporarily remove peer cross dependencies (#2123)
* chore: temporarily remove peer cross dependencies

* up lock

* add missing dep
2024-09-06 00:44:59 +02:00
Sasha
5d3cc5fd45
fix: filter missing messages (#2119)
* bug: fix filter missing messages

* fix keep alive

* fix const
2024-09-03 00:59:21 +02:00
Sasha
5cfe9327b6
chore: remove relay from sdk package (#2040)
* remove full node

* remove relay from sdk

* comment node counter check

* try using logline

* up comment
2024-08-31 15:18:51 +02:00
Sasha
f9361bf774
chore: release master (#2066) proto-v0.0.8 discovery-v0.0.4 message-hash-v0.1.15 message-encryption-v0.0.29 utils-v0.0.19 relay-v0.0.14 core-v0.0.31 enr-v0.0.25 sdk-v0.0.27 interfaces-v0.0.26 2024-08-29 11:49:04 +02:00
Sasha
a022433851
feat(lightpush)!: return new error messages (#2115)
* feat: return new error messages

* fix test, remove unused

* up

* up

* rollback

* up test
2024-08-29 11:20:19 +02:00
Sasha
eadb85ab83
feat: offline state recovery for Filter subscription (#2049)
* up

* fix window reference

* add tests

* up

* add e2e renew test

* address comments

* remove unused

* add test

* try

* remove only

* up test

* up

* remove only

* add tmp logs, use before/after hooks

* up

* fix check

* remove only

* fix test

* up
2024-08-28 18:00:18 +02:00
Sasha
71384dfdfd
chore: change amount of used peers to 2 (#2111)
* chore: change amount of used peers to 2

* up

* up

* up tests
2024-08-27 21:53:16 +02:00
chair
f0a5c6f056
chore: update add-action-project.yml (#2114) 2024-08-27 11:11:35 +02:00
Danish Arora
69b6612ec9
chore: upgrade node to LTS 20 (#2110)
* chore: upgrade nodejs to new LTS 20

* chore: update other references
2024-08-23 15:57:20 +05:30
Sasha
8f56d90cf1
fix: import of base_protocol and networkConfig type guard (#2109)
* bug: fix import

* fix type guard
2024-08-23 11:44:55 +02:00
Sasha
c33844eddf
chore: update husky to new api (#2102) 2024-08-22 18:01:29 +02:00