3133 Commits

Author SHA1 Message Date
Sasha
2fba052b8b
feat: add start/stop to filter (#2592) 2025-08-30 01:32:59 +02:00
fryorcraken
914beb6531
fix: (sds) ensure incoming messages have their retrieval hint stored (#2604)
* SDS: export `MessageId`

* SDS: attach retrieval hints to incoming messages

* sds: ensure items are ordered by timestamp

* test: sds: avoid using "as any" as it bypasses type checks

* test: filter: avoid using "as any" as it bypasses type checks

* test: fix tests without introducing proxy
2025-08-28 15:57:23 +10:00
fryorcraken
8542d04bf5
feat: query on connect (#2602)
* feat: query on connect

Perform store time-range queries upon connecting to a store node.
Some heuristics are applied to ensure the store queries are not too frequent.

* make `maybeQuery` private

* query-on-connect: use index.ts only for re-export

* query-on-connect: update doc
2025-08-28 13:58:16 +10:00
fryorcraken
826aedd558
log: logs are already prefixed with waku (#2603) 2025-08-27 23:32:41 +10:00
fryorcraken
0dfbcf6b6b
feat: StoreConnect events (#2601)
* store connect evt: use enum instead of free strings for Waku event types

* store connect evt: more accurate enum name

* store connect evt: add store connect event on peer manager

* store connect evt: simplify logic statements

* store connect evt: test store connect

* store connect evt: export event types

* test: use enum

* Shorter name for waku events
2025-08-27 12:29:22 +10:00
fryorcraken
78c856d079
fix: remove sharding circular dependency (#2590) 2025-08-25 10:49:05 +10:00
Sasha
836d6b8793
feat: expose message hash from IDecodedMessage (#2578)
* expose message hash from IDecodedMessage

* up mock

* optimize hashing and add tests
2025-08-21 11:58:43 +02:00
Sasha
c8dfdb1ace
fix: make health events emission consistent (#2570)
* fix health events emission

* fix bloom test flakiness due to bloom filter properties
2025-08-20 13:13:38 +02:00
Sasha
26de2d11c8
chore: release master (#2515) interfaces-v0.0.33 message-encryption-v0.0.36 relay-v0.0.21 enr-v0.0.32 utils-v0.0.26 proto-v0.0.13 discovery-v0.0.11 sds-v0.0.6 rln-v0.1.8 core-v0.0.38 sdk-v0.0.34 2025-08-15 21:56:59 +02:00
Sasha
dfb2baf004
fix: prevent setting shard info from PX if it exists (#2561)
* prevent setting px shard info

* address review
2025-08-15 00:30:07 +02:00
Sasha
eab8ce81b4
feat!: local peer discovery improvements (#2557)
* update local peer discovery, make it configurable for cache

* move to separate file

* up tests, remove local storage from tests

* pass local peer cache options

* add e2e tests

* add aditional e2e tests for local cache

* rename local-peer-cache into peer-cache

* update tests, ci

* prevent filterign ws addresses
2025-08-15 00:14:32 +02:00
Sasha
95da57a870
feat: peer exchange discovery improvements (#2537)
* rename, mark stuff to remove

* rename type

* update exports from discovery package, improve PX implementation

* re-structure px discovery, comment tests

* implement UT, E2E Tests, clean code a bit, implement recurring PX discovery, move DNS E2E Tests to separate folder, remove not needed E2E tests

* fix discovery dialer e2e test

* mark as started

* fix dns tests

* attempt to fix e2e

* skip test

* update tests

* fix typo

* add catch all in stream manager, update tests

* update mock

* update test
2025-08-14 09:47:55 +02:00
fryorcraken
c161b37d08
fix!: SDS acknowledgements (#2549)
* SDS: change default causal history size to 200

# Conflicts:
#	packages/sds/src/index.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS: add some comments

* SDS: segregate messages types, introduce LocalHistory

* SDS: fix miss-acks

* SDS: logs and more explicit variable names

* SDS: shorten event name

* SDS: shorten var name

* SDS: move message classes to own file.

* SDS: use lodash instead of custom SortedArray implementation

* SDS: move Message to own file

* SDS: add comparison tests
2025-08-14 10:44:18 +10:00
fryorcraken
de972d6694
fix: reduce emission of health events (#2554) 2025-08-13 23:42:22 +10:00
Sasha
ada265731a
fix: improve error handling for stream manager (#2546)
* fix: improve error handling for stream manager

* fix browser tests

* update logs for stream manager

* fix tests

* fix playwright
2025-08-12 23:25:23 +02:00
fryorcraken
dc5155056b
feat!: SDS improvements and fixes (#2539)
* introduce `MessageId` type

# Conflicts:
#	packages/sds/src/message_channel/message_channel.ts

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* doc: long term solution is SDS protocol change

* SDS: renaming to match message function

* SDS: introduce `Message` class for easier encoding/decoding

# Conflicts:
#	packages/sds/src/message_channel/events.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS Message is a class now

* SDS: it's "possibly" not "partially" acknowledged.

* SDS: TODO

* SDS: fix tests

* SDS: make logs start with `waku`

* SDS: add bloom filter test

# Conflicts:
#	packages/sds/src/message_channel/events.spec.ts

* SDS: improve naming

* SDS: improve naming

Messages are not "sent" or received, but pushed for processing in local queues.

* SDS: sync message should not be delivered

* SDS: renaming from earlier

* SDS: remove useless variable

* SDS: Fix comment

* SDS: sync messages do not get "delivered"

* SDS: acks

* SDS: simplify delivered event

* SDS: improve event naming

* SDS: fix comment

* SDS: make task error an official event

* SDS: Mark messages that are irretrievably lost

* SDS: remove default for irretrievable and simplify config

* SDS: typo on sync event

* SDS: add and user sender id

* SDS: resent message never get ack'd

* SDS: fix cylic dependencies

* SDS: helpful logs

* SDS: avoid duplicate history entries

* SDS: export options
2025-08-12 10:47:52 +10:00
fryorcraken
459fe96fe6
fix: should not self-acknowledge messages (#2528)
* introduce `MessageId` type

* fix deprecated import

* test: own messages are not used for acks

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* test: do not self-possible-ack

* doc: long term solution is SDS protocol change
2025-08-08 10:18:01 +10:00
fryorcraken
52e5c34520
test: add browser tests for sds (#2527)
* tests: add browser tests for sds

* test: use FIREFOX_BIN if set
2025-08-06 10:30:00 +10:00
fryorcraken
449797d5c1
fix: bloom filter protobuf decoding (#2529)
* SDS: add bloom filter test

* fix: ensure bloom filter can be decoded from protobuf
2025-08-05 14:55:57 +10:00
Arseniy Klempner
103f21ef85
Merge pull request #2313 from waku-org/fix/do-not-limit-dns-discovered-peers
feat!: do not limit DNS Peer Discovery on capability
2025-07-31 17:06:23 -07:00
fryorcraken
d7919e8c9b
improve var name 2025-07-29 19:29:21 +10:00
fryorcraken
f4a2778e02
test: return first peer without traversing branch 2025-07-29 19:29:20 +10:00
fryorcraken
25f884e05b
feat: retrieve peers from all passed enrtree URLs
Order retrieval is random, but it attemps to retrieve from all ENR trees.
2025-07-29 19:29:20 +10:00
fryorcraken
0dfe35281c
fix: do not limit DNS Peer Discovery on capability
Nodes returned by DNS Discovery are not guaranteed to be reachable.
Hence, setting an upfront limit based on sought capability does not provide
any guarantees that such capability should be reached.

The discovery's mechanism role is to find Waku 2 nodes. As many as possible.

It is then the role of the peer manager to decide:
- whether to attempt connecting to the node based on advertised capabilities
- retain connection to the node based on actual mounted protocols.

We still want to prevent infinite loops, hence the `maxGet` parameter.
Also, there was a dichotomy between code tested, and code actually used by libp2p peer discovery, now resolved.

# Conflicts:
#	packages/discovery/src/dns/constants.ts
2025-07-29 19:29:13 +10:00
fbarbu15
c1f9471cd7
chore: extra reliability tests 2 (#2450)
* chore: add throughput reliability tests

* chore: add capability to run all

* chore: add network-latency reliability tests

* chore: add network-latency reliability tests

* chore: add other network reliability tests

* chore: add other network reliability tests

* chore: fix tc cleanup

* chore: refactor common code

* chore: refactor common code

* chore: refactor common code

* chore: refactor common code

* chore: refactor common code

* chore: refactor common code

* chore: refactor common code

* chore: fix

* chore: fix tests

* chore: fix tests
2025-07-29 10:17:43 +03:00
fryorcraken
0be4861c79
Merge pull request #2471 from waku-org/waku-api/fix-nwaku-master
feat!: Introduce routing info concept
2025-07-22 11:37:53 +10:00
fryorcraken
f8f20db85c
small optimization 2025-07-22 10:59:47 +10:00
fryorcraken
cd265ba1ae
fix test for nwaku master 2025-07-21 15:54:05 +10:00
fryorcraken
39c139158c
Delete dual test (makes it hard to work on nwaku master) 2025-07-21 15:44:30 +10:00
fryorcraken
77c694095d
fixing test against nwaku nightly 2025-07-21 15:40:05 +10:00
fryorcraken
158f6ecf98
fix retry_manager test 2025-07-21 15:02:26 +10:00
fryorcraken
d4429702c2
introduce interface for better mocking
As one could fall in a trap of not defining the right methods on the mock
2025-07-21 14:47:07 +10:00
fryorcraken
1e7e0291fa
separate file tests
To not have 2 `runTests` in same file.
2025-07-21 14:10:00 +10:00
fryorcraken
b51d598581
re-enable commented test 2025-07-21 13:57:06 +10:00
fryorcraken
ec8ded9c6a
revert distracting changes 2025-07-21 12:26:51 +10:00
fryorcraken
3bb5a4931f
fix peer exchange test 2025-07-21 12:26:29 +10:00
fryorcraken
0739fd1dd2
revert shard info -> relay shard renaming 2025-07-19 21:50:51 +10:00
fryorcraken
e5919a6bd9
revert most invasive changes 2025-07-19 21:25:21 +10:00
fryorcraken
97a26b2373
revert test changes 2025-07-19 19:21:39 +10:00
fryorcraken
e53717cd08
revert temporary change 2025-07-19 19:03:51 +10:00
fryorcraken
3c930dedfc
revert change on encoder 2025-07-19 18:07:38 +10:00
fryorcraken
065ef0adb0
getRoutingInfo is best as a method as it can use the context from this. 2025-07-19 17:32:58 +10:00
fryorcraken
8d278f8132
revert change that is just noise 2025-07-19 17:24:34 +10:00
fryorcraken
1bbb129d04
fix relay test 2025-07-19 17:19:58 +10:00
fryorcraken
3200b19a02
Simplify IRouting interface 2025-07-19 14:45:06 +10:00
fryorcraken
d04130feb6
use IRouting info for typing 2025-07-19 14:30:44 +10:00
fryorcraken
b8867dee38
test: re-introduce usage of ensureSubscriptions 2025-07-19 14:24:30 +10:00
fryorcraken
b4787e0e87
remove default values for cluster id and num shards in cluster
These are high level configuration values, that need to be consistent within a given execution. Setting them as default value at such a low level is a footgun risk.
2025-07-19 13:22:45 +10:00
fryorcraken
3842d84b55
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).

This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.

It also included various back and forth conversions between shards, pubsub topics, etc.

With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.

Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.

# Conflicts:
#	packages/core/src/lib/connection_manager/connection_manager.ts
#	packages/core/src/lib/metadata/metadata.ts
#	packages/interfaces/src/metadata.ts
#	packages/interfaces/src/sharding.ts
#	packages/relay/src/create.ts
#	packages/sdk/src/filter/filter.ts
#	packages/sdk/src/filter/types.ts
#	packages/sdk/src/light_push/light_push.spec.ts
#	packages/tests/tests/sharding/auto_sharding.spec.ts
#	packages/tests/tests/sharding/static_sharding.spec.ts

# Conflicts:
#	packages/sdk/src/store/store.ts
2025-07-19 13:22:45 +10:00
Arseniy Klempner
e92cc570f2
Merge pull request #2441 from waku-org/release-please--branches--master
chore: release master
core-v0.0.37 interfaces-v0.0.32 rln-v0.1.7 sdk-v0.0.33 message-encryption-v0.0.35 discovery-v0.0.10 proto-v0.0.12 utils-v0.0.25 sds-v0.0.5 enr-v0.0.31 relay-v0.0.20
2025-07-18 18:54:31 -07:00