25 Commits

Author SHA1 Message Date
fryorcraken
914beb6531
fix: (sds) ensure incoming messages have their retrieval hint stored (#2604)
* SDS: export `MessageId`

* SDS: attach retrieval hints to incoming messages

* sds: ensure items are ordered by timestamp

* test: sds: avoid using "as any" as it bypasses type checks

* test: filter: avoid using "as any" as it bypasses type checks

* test: fix tests without introducing proxy
2025-08-28 15:57:23 +10:00
fryorcraken
826aedd558
log: logs are already prefixed with waku (#2603) 2025-08-27 23:32:41 +10:00
Sasha
c8dfdb1ace
fix: make health events emission consistent (#2570)
* fix health events emission

* fix bloom test flakiness due to bloom filter properties
2025-08-20 13:13:38 +02:00
Sasha
26de2d11c8
chore: release master (#2515) 2025-08-15 21:56:59 +02:00
fryorcraken
c161b37d08
fix!: SDS acknowledgements (#2549)
* SDS: change default causal history size to 200

# Conflicts:
#	packages/sds/src/index.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS: add some comments

* SDS: segregate messages types, introduce LocalHistory

* SDS: fix miss-acks

* SDS: logs and more explicit variable names

* SDS: shorten event name

* SDS: shorten var name

* SDS: move message classes to own file.

* SDS: use lodash instead of custom SortedArray implementation

* SDS: move Message to own file

* SDS: add comparison tests
2025-08-14 10:44:18 +10:00
fryorcraken
dc5155056b
feat!: SDS improvements and fixes (#2539)
* introduce `MessageId` type

# Conflicts:
#	packages/sds/src/message_channel/message_channel.ts

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* doc: long term solution is SDS protocol change

* SDS: renaming to match message function

* SDS: introduce `Message` class for easier encoding/decoding

# Conflicts:
#	packages/sds/src/message_channel/events.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS Message is a class now

* SDS: it's "possibly" not "partially" acknowledged.

* SDS: TODO

* SDS: fix tests

* SDS: make logs start with `waku`

* SDS: add bloom filter test

# Conflicts:
#	packages/sds/src/message_channel/events.spec.ts

* SDS: improve naming

* SDS: improve naming

Messages are not "sent" or received, but pushed for processing in local queues.

* SDS: sync message should not be delivered

* SDS: renaming from earlier

* SDS: remove useless variable

* SDS: Fix comment

* SDS: sync messages do not get "delivered"

* SDS: acks

* SDS: simplify delivered event

* SDS: improve event naming

* SDS: fix comment

* SDS: make task error an official event

* SDS: Mark messages that are irretrievably lost

* SDS: remove default for irretrievable and simplify config

* SDS: typo on sync event

* SDS: add and user sender id

* SDS: resent message never get ack'd

* SDS: fix cylic dependencies

* SDS: helpful logs

* SDS: avoid duplicate history entries

* SDS: export options
2025-08-12 10:47:52 +10:00
fryorcraken
459fe96fe6
fix: should not self-acknowledge messages (#2528)
* introduce `MessageId` type

* fix deprecated import

* test: own messages are not used for acks

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* test: do not self-possible-ack

* doc: long term solution is SDS protocol change
2025-08-08 10:18:01 +10:00
fryorcraken
52e5c34520
test: add browser tests for sds (#2527)
* tests: add browser tests for sds

* test: use FIREFOX_BIN if set
2025-08-06 10:30:00 +10:00
fryorcraken
449797d5c1
fix: bloom filter protobuf decoding (#2529)
* SDS: add bloom filter test

* fix: ensure bloom filter can be decoded from protobuf
2025-08-05 14:55:57 +10:00
Sasha
79f319cc86 chore: release master 2025-07-19 01:52:31 +02:00
Sasha
cd1d909de3
chore: release master (#2390) 2025-06-23 22:29:40 +02:00
Sasha
fcc6496fef
chore!: upgrade libp2p, nodejs and typescript (#2401)
* remove react native package

* upgrade nodejs

* upgrade libp2p

* upgrade typescript, use 22 node

* up lock

* fix node compatibility

* upgrade playwright

* upgrade proto

* update tests
2025-06-22 17:41:23 +02:00
Arseniy Klempner
8444bc940f
fix: reorder methods by visibility 2025-06-13 15:22:52 -07:00
Arseniy Klempner
a2c3b2e6aa
feat(sds)_: add command queue architecture and improve message handling
- Introduce command queue system for sequential task processing
- Add comprehensive event system for message lifecycle tracking
- Restructure codebase with separate bloom_filter directory
- Export encode/decode helpers for SDS proto messages
- Use Set for deduplication in missing message detection
- Fix sync message handling for empty content messages
- Always emit MissedMessages event even with empty array
- Improve duplicate message detection logic
2025-06-10 20:19:22 -07:00
Sasha
1905558753
feat!: re-work messaging parts and sharding (#2399)
* remove message-hash package, move it to core

* add js-doc to message hash

* up

* address type changes

* fix lint
2025-06-03 11:08:02 +02:00
Sasha
e22f211d46 chore: release master 2025-04-23 07:38:08 +02:00
Arseniy Klempner
408be95a13
feat(sds): add retrieval hint to causal history 2025-04-22 07:45:05 -07:00
Arseniy Klempner
6b4848c853
feat(sds): adds ephemeral messages, delivered message callback and event 2025-04-22 07:16:14 -07:00
Sasha
ea6daae927
chore: release master (#2311) 2025-03-24 14:41:33 +01:00
Arseniy Klempner
13ae5d4f73
feat(sds): send and receive sync messages 2025-03-13 11:29:55 -07:00
Arseniy Klempner
f7666a6588
feat(sds): adds logic to sweep incoming and outgoing buffers 2025-03-03 22:09:44 -08:00
Arseniy Klempner
389ca4062e
feat(sds): add message channel with buffers and send/receive logic
This commit creates the class for an SDS message channel, including
buffers for outgoing and incoming messages. Adds logic for sending
messages, receiving messages, delivering messages, and reviewing
acknowledgement status of messages. Also adds byte serialization
for bloom filters.
2025-02-21 14:12:29 -08:00
Danish Arora
becb46f3a5
chore(sds): use allure reporting (#2255) 2025-02-11 00:56:00 +05:30
Arseniy Klempner
be93e4b71f
feat(sds): migrate bloomfilter to bigint and import hashn function from nim
Uses an array of bigint to store sufficient bits in bloom filter.
Updates all arithmetic to explicitly cast to bigint where necessary.
Makes the hashn function for bloomfilter a parameter.
Adds an implementation of hashn generated using nim compiler.
Adds tests.
2025-02-05 17:27:41 -08:00
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00