12 Commits

Author SHA1 Message Date
Hanno Cornelius
5334a7fcc9
feat: add SDS-Repair (SDS-R) to the SDS implementation (#2698)
* wip

* feat: integrate sds-r with message channels

* feat: add SDS-R events

* fix: fixed buffer handling incoming and outgoing

* fix: more buffer fixes

* fix: remove some magic numbers

* fix: buffer optimisation, backwards compatible senderId

* fix: fix implementation guide, remove unrelated claude file

* fix: further buffer optimisations

* fix: linting errors

* fix: suggestions from code review

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* fix: remove implementation guide

* fix: build errors, remove override, improve buffer

* fix: consistent use of MessageId and ParticipantId

* fix: switch to conditionally constructed from conditionally executed

---------

Co-authored-by: fryorcraken <commits@fryorcraken.xyz>
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2025-10-28 10:27:06 +00:00
fryorcraken
c0ecb6abba
fix!: SDS lamport timestamp overflow and keep it to current time (#2664)
* fix!: avoid SDS lamport timestamp overflow

The SDS timestamp is initialized to the current time in milliseconds, which is a 13 digits value (e.g. 1,759,223,090,052).

The maximum value for int32 is 2,147,483,647 (10 digits), which is clearly less than the timestamp.
Maximum value for uint32 is 4,294,967,295 (10 digits), which does not help with ms timestamp.

uint64 is BigInt in JavaScript, so best to be avoided unless strictly necessary as it creates complexity.
max uint64 is 18,446,744,073,709,551,615 (20 digits).

Using seconds instead of milliseconds would enable usage of uint32 valid until the year 2106.

The lamport timestamp is only initialized to current time for a new channel. The only scenario is when a user comes in a channel, and thinks it's new (did not get previous messages), and then starts sending messages. Meaning that there may be an initial timestamp conflict until the logs are consolidated, which is already handled by the protocol.

* change lamportTimestamp to uint64 in protobuf

* lamport timestamp remains close to current time
2025-10-02 09:07:10 +10:00
fryorcraken
dc5155056b
feat!: SDS improvements and fixes (#2539)
* introduce `MessageId` type

# Conflicts:
#	packages/sds/src/message_channel/message_channel.ts

* fix: own messages are not used for ack

* fix: own messages are not used for ack

* doc: long term solution is SDS protocol change

* SDS: renaming to match message function

* SDS: introduce `Message` class for easier encoding/decoding

# Conflicts:
#	packages/sds/src/message_channel/events.ts
#	packages/sds/src/message_channel/message_channel.ts

* SDS Message is a class now

* SDS: it's "possibly" not "partially" acknowledged.

* SDS: TODO

* SDS: fix tests

* SDS: make logs start with `waku`

* SDS: add bloom filter test

# Conflicts:
#	packages/sds/src/message_channel/events.spec.ts

* SDS: improve naming

* SDS: improve naming

Messages are not "sent" or received, but pushed for processing in local queues.

* SDS: sync message should not be delivered

* SDS: renaming from earlier

* SDS: remove useless variable

* SDS: Fix comment

* SDS: sync messages do not get "delivered"

* SDS: acks

* SDS: simplify delivered event

* SDS: improve event naming

* SDS: fix comment

* SDS: make task error an official event

* SDS: Mark messages that are irretrievably lost

* SDS: remove default for irretrievable and simplify config

* SDS: typo on sync event

* SDS: add and user sender id

* SDS: resent message never get ack'd

* SDS: fix cylic dependencies

* SDS: helpful logs

* SDS: avoid duplicate history entries

* SDS: export options
2025-08-12 10:47:52 +10:00
Arseniy Klempner
94788d0dfc
feat(proto): add light push v3 definition 2025-07-14 08:39:41 -07:00
Sasha
fcc6496fef
chore!: upgrade libp2p, nodejs and typescript (#2401)
* remove react native package

* upgrade nodejs

* upgrade libp2p

* upgrade typescript, use 22 node

* up lock

* fix node compatibility

* upgrade playwright

* upgrade proto

* update tests
2025-06-22 17:41:23 +02:00
Arseniy Klempner
408be95a13
feat(sds): add retrieval hint to causal history 2025-04-22 07:45:05 -07:00
Arseniy Klempner
468512fa85
feat(sds): create package for sds and add protobuf def
Adds a new package for the browser implementation of scalable data
sync. Ports some of the nim implementation of bloom filter to ts.
Adds protobuf definition for SDS messages.
2025-01-28 18:53:41 -08:00
Danish Arora
409642d7d3
chore: upgrade libp2p and related packages (#2084)
* chore: upgrade libp2p and related packages

* fix: public key on peer id can now be undefined

* chore: upgrade remainder packages

* chore: fix tests

* chore: address comments

* chore: upgrade playwright and proto
2024-08-14 04:23:04 +05:30
Danish Arora
86f730f958
feat!: store v3 (#2036)
* feat: write proto

* chore: move store v2 to a subdir

* chore: update v3 proto

* feat: create custom RPC

* feat: implement storev3 core

* chore: set store v3 as default

* chore: move v2 related code

* chore: update v2 imports

* feat: add store-v3 sdk implementation

* fix: rebase

* chore: add ts-doc for store query request params

* chore: update tests for new API

* fix: use nanoseconds instead of millisecond for timerange

* chore: improve store

* chore: remove store v2

* chore: update tests

* chore: fix legacy imports & proto

* tests: remove manual reversal as its part of the API, update incorrect cursor error msg

* chore: update default page size

* chore: account for MAX_PAGE_SIZE from nwaku

* fix: test

* fix: sorting tests
2024-08-06 12:06:37 +05:30
Arseniy Klempner
6990e6f4e0
fix: use correct error message for store rate limit 2024-04-24 15:00:26 -07:00
NagyZoltanPeter
aa3ce1fc40
Added missing values for HistoryError to be sync with nwaku 2024-04-09 15:11:20 +02:00
Danish Arora
8ec11b2cde
chore: bump protons (#1898)
* regen lockfile

* commit new proto code

* chore: upgrade protons-runtime to 5.4.0

* chore: have src/generated as a dedicated directory for generated proto code

* chore: ignore unused vars in proto

* chore: update lockfile

* fix: eslint errors

* chore: handle flaky tests edge case

* chore: update playright docker image

* rm: only

* rm: console

* chore: regen lockfile

* chore: update playright docker image
2024-04-07 10:55:09 +03:00