6 Commits

Author SHA1 Message Date
Hanno Cornelius
5334a7fcc9
feat: add SDS-Repair (SDS-R) to the SDS implementation (#2698)
* wip

* feat: integrate sds-r with message channels

* feat: add SDS-R events

* fix: fixed buffer handling incoming and outgoing

* fix: more buffer fixes

* fix: remove some magic numbers

* fix: buffer optimisation, backwards compatible senderId

* fix: fix implementation guide, remove unrelated claude file

* fix: further buffer optimisations

* fix: linting errors

* fix: suggestions from code review

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* fix: remove implementation guide

* fix: build errors, remove override, improve buffer

* fix: consistent use of MessageId and ParticipantId

* fix: switch to conditionally constructed from conditionally executed

---------

Co-authored-by: fryorcraken <commits@fryorcraken.xyz>
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2025-10-28 10:27:06 +00:00
fryorcraken
914beb6531
fix: (sds) ensure incoming messages have their retrieval hint stored (#2604)
* SDS: export `MessageId`

* SDS: attach retrieval hints to incoming messages

* sds: ensure items are ordered by timestamp

* test: sds: avoid using "as any" as it bypasses type checks

* test: filter: avoid using "as any" as it bypasses type checks

* test: fix tests without introducing proxy
2025-08-28 15:57:23 +10:00
fryorcraken
c161b37d08
fix!: SDS acknowledgements (#2549)
* SDS: change default causal history size to 200

# Conflicts:
#	packages/sds/src/index.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS: add some comments

* SDS: segregate messages types, introduce LocalHistory

* SDS: fix miss-acks

* SDS: logs and more explicit variable names

* SDS: shorten event name

* SDS: shorten var name

* SDS: move message classes to own file.

* SDS: use lodash instead of custom SortedArray implementation

* SDS: move Message to own file

* SDS: add comparison tests
2025-08-14 10:44:18 +10:00
fryorcraken
dc5155056b
feat!: SDS improvements and fixes (#2539)
* introduce `MessageId` type

# Conflicts:
#	packages/sds/src/message_channel/message_channel.ts

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* doc: long term solution is SDS protocol change

* SDS: renaming to match message function

* SDS: introduce `Message` class for easier encoding/decoding

# Conflicts:
#	packages/sds/src/message_channel/events.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS Message is a class now

* SDS: it's "possibly" not "partially" acknowledged.

* SDS: TODO

* SDS: fix tests

* SDS: make logs start with `waku`

* SDS: add bloom filter test

# Conflicts:
#	packages/sds/src/message_channel/events.spec.ts

* SDS: improve naming

* SDS: improve naming

Messages are not "sent" or received, but pushed for processing in local queues.

* SDS: sync message should not be delivered

* SDS: renaming from earlier

* SDS: remove useless variable

* SDS: Fix comment

* SDS: sync messages do not get "delivered"

* SDS: acks

* SDS: simplify delivered event

* SDS: improve event naming

* SDS: fix comment

* SDS: make task error an official event

* SDS: Mark messages that are irretrievably lost

* SDS: remove default for irretrievable and simplify config

* SDS: typo on sync event

* SDS: add and user sender id

* SDS: resent message never get ack'd

* SDS: fix cylic dependencies

* SDS: helpful logs

* SDS: avoid duplicate history entries

* SDS: export options
2025-08-12 10:47:52 +10:00
Arseniy Klempner
a2c3b2e6aa
feat(sds)_: add command queue architecture and improve message handling
- Introduce command queue system for sequential task processing
- Add comprehensive event system for message lifecycle tracking
- Restructure codebase with separate bloom_filter directory
- Export encode/decode helpers for SDS proto messages
- Use Set for deduplication in missing message detection
- Fix sync message handling for empty content messages
- Always emit MissedMessages event even with empty array
- Improve duplicate message detection logic
2025-06-10 20:19:22 -07:00
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00