5 Commits

Author SHA1 Message Date
Hanno Cornelius
5334a7fcc9
feat: add SDS-Repair (SDS-R) to the SDS implementation (#2698)
* wip

* feat: integrate sds-r with message channels

* feat: add SDS-R events

* fix: fixed buffer handling incoming and outgoing

* fix: more buffer fixes

* fix: remove some magic numbers

* fix: buffer optimisation, backwards compatible senderId

* fix: fix implementation guide, remove unrelated claude file

* fix: further buffer optimisations

* fix: linting errors

* fix: suggestions from code review

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* fix: remove implementation guide

* fix: build errors, remove override, improve buffer

* fix: consistent use of MessageId and ParticipantId

* fix: switch to conditionally constructed from conditionally executed

---------

Co-authored-by: fryorcraken <commits@fryorcraken.xyz>
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2025-10-28 10:27:06 +00:00
fryorcraken
c0ecb6abba
fix!: SDS lamport timestamp overflow and keep it to current time (#2664)
* fix!: avoid SDS lamport timestamp overflow

The SDS timestamp is initialized to the current time in milliseconds, which is a 13 digits value (e.g. 1,759,223,090,052).

The maximum value for int32 is 2,147,483,647 (10 digits), which is clearly less than the timestamp.
Maximum value for uint32 is 4,294,967,295 (10 digits), which does not help with ms timestamp.

uint64 is BigInt in JavaScript, so best to be avoided unless strictly necessary as it creates complexity.
max uint64 is 18,446,744,073,709,551,615 (20 digits).

Using seconds instead of milliseconds would enable usage of uint32 valid until the year 2106.

The lamport timestamp is only initialized to current time for a new channel. The only scenario is when a user comes in a channel, and thinks it's new (did not get previous messages), and then starts sending messages. Meaning that there may be an initial timestamp conflict until the logs are consolidated, which is already handled by the protocol.

* change lamportTimestamp to uint64 in protobuf

* lamport timestamp remains close to current time
2025-10-02 09:07:10 +10:00
fryorcraken
dc5155056b
feat!: SDS improvements and fixes (#2539)
* introduce `MessageId` type

# Conflicts:
#	packages/sds/src/message_channel/message_channel.ts

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* doc: long term solution is SDS protocol change

* SDS: renaming to match message function

* SDS: introduce `Message` class for easier encoding/decoding

# Conflicts:
#	packages/sds/src/message_channel/events.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS Message is a class now

* SDS: it's "possibly" not "partially" acknowledged.

* SDS: TODO

* SDS: fix tests

* SDS: make logs start with `waku`

* SDS: add bloom filter test

# Conflicts:
#	packages/sds/src/message_channel/events.spec.ts

* SDS: improve naming

* SDS: improve naming

Messages are not "sent" or received, but pushed for processing in local queues.

* SDS: sync message should not be delivered

* SDS: renaming from earlier

* SDS: remove useless variable

* SDS: Fix comment

* SDS: sync messages do not get "delivered"

* SDS: acks

* SDS: simplify delivered event

* SDS: improve event naming

* SDS: fix comment

* SDS: make task error an official event

* SDS: Mark messages that are irretrievably lost

* SDS: remove default for irretrievable and simplify config

* SDS: typo on sync event

* SDS: add and user sender id

* SDS: resent message never get ack'd

* SDS: fix cylic dependencies

* SDS: helpful logs

* SDS: avoid duplicate history entries

* SDS: export options
2025-08-12 10:47:52 +10:00
Arseniy Klempner
408be95a13
feat(sds): add retrieval hint to causal history 2025-04-22 07:45:05 -07:00
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00