logos-messaging/specs
1
0
Fork 0
mirror of https://github.com/logos-messaging/specs.git synced 2026-01-04 07:03:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

waku sync 2.0 initial draft

Browse Source
This commit is contained in:
SionoiS 2024-12-05 11:34:36 -05:00
parent b38f248b4f
commit cba8da96bf
No known key found for this signature in database
GPG Key ID: C9458A8CB1852951
1 changed files with 108 additions and 85 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

193
standards/core/sync.md
View File

@ -11,78 +11,131 @@ contributors:
This specification explains the `WAKU-SYNC` protocol This specification explains the `WAKU-SYNC` protocol
which enables the reconciliation of two sets of message hashes which enables the reconciliation of two sets of message hashes
in the context of keeping multiple Store nodes synchronized. in the context of keeping multiple Store nodes synchronized.
Waku Sync is a wrapper around
[Negentropy](https://github.com/hoytech/negentropy) a [range-based set reconciliation protocol](https://logperiodic.com/rbsr.html).
## Specification ## Specification
**Protocol identifier**: `/vac/waku/sync/1.0.0` **Protocol identifier**: `/vac/waku/reconciliation/1.0.0`
### Terminology #### Terminology
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt). “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
The term Negentropy refers to the protocol of the same name. #### Message Ids
Negentropy payload refers to Message Ids MUST be composed of the timestamp and the hash of the Waku messages.
the messages created by the Negentropy protocol.
Client always refers to the initiator
and the server the receiver of the first payload.
### Design Requirements The timestamp MUST be the time of creation and
Nodes enabling Waku Sync SHOULD the hash MUST follow the
manage and keep message hashes in a local cache [deterministic message hashing specification](https://rfc.vac.dev/waku/standards/core/14/message#deterministic-message-hashing)
for the range of time
during which synchronization is required.
Nodes SHOULD use the same time range,
for Waku we chose one hour as the global default.
Waku Relay or Light Push protocol MAY be enabled
and used in conjunction with Sync
as a source of new message hashes
for the cache.
Nodes MAY use the Store protocol > This way the message Ids can always be totally ordered.
to request missing messages once reconciliation is complete Chronologically according to the timestamp and
or to provide messages to requesting clients. disambiguate based on the hash lexical order
in cases where the timestamp is the same.
### Payload #### Range Bounds
A range MUST consists of 2 Id bounds, the first bound is
inclusive the second bound exclusive.
The first bound MUST be strictly smaller than the second one.
```protobuf #### Range Fingerprinting
syntax = "proto3"; The fingerprint of a range MUST be the XOR operation applied to
all the hashes of the Ids included in that range.
package waku.sync.v1; #### Range Type
Every range MUST have one of the following types; skip, fingerprint or item set.
message SyncPayload { - Skip type is used to signal already processed ranges that MUST be ignored.
optional bytes negentropy = 1; - Fingerprint type signify that this range fingerprint MUST be compared when received.
- Item set type contain multiple message Ids that MUST all be compared when received.
> Item sets are an optimization, stopping the recursion early can
save many network roundtrips.
repeated bytes hashes = 20; #### Range Processing
} Ranges have to be processed differently acording to their types.
```
### Session Flow - Skip ranges MUST be merged with other consequtive ones if possible.
A client initiates a session with a server - Equal fingerprint ranges MUST become skip ranges.
by sending a `SyncPayload` with - Unequal fingerprint ranges MUST be splitted into smaller ranges. The new type MAY be either fingerprint or item set.
only the `negentropy` field set. - Unresolved item set ranges MUST be checked for differences and marked resolved.
This field MUST contain - Resolved item set ranges MUST be checked for differences and become skip ranges.
the first negentropy payload
created by the client
for this session.
The server receives a `SyncPayload`. ### Delta Encoding
A new negentropy payload is computed from the received one. For efficient transmition of timestamps, hashes and ranges. Payloads are delta encoded as follow.
The server sends back a `SyncPayload` to the client.
The client receives a `SyncPayload`. All ranges to be transmitted MUST be ordered and only upper bounds used.
A new negentropy payload OR an empty one is computed. > Inclusive lower bounds can be omitted because they are always
If a new payload is computed then the same as the exclusive upper bounds of the previous range or zero.
the exchanges between client and server continues until
the client computes an empty payload. To achieve this, it MAY be needed to add skip ranges.
This client computation also outputs any hash differences found, > For example, a skip range can be added with
those MUST be stored. an exclusive upper bound equal to the first range lower bound.
In the case of an empty payload, This way the receiving peer knows to ignore the range from zero to the start of the ranges
the reconciliation is done,
the client MUST send back a `SyncPayload` Every timestamps after the first MUST be noted as the difference from the previous one.
with all the missing server hashes in the `hashes` field and If the timestamp is the same, zero MUST be used and the hash MUST be added.
an empty `nengentropy` field. The added hash MUST be trucated up to and including the first differetiating byte.
| Timestamp | Hash | Timestamp (encoded) | Hash (encoded)
| - | - | - | -
| 1000 | 0x4a8a769a... | 1000 | -
| 1002 | 0x351c5e86... | 2 | -
| 1002 | 0x3560d9c4... | 0 | 0x3560
| 1003 | 0xbeabef25... | 1 | -
#### Varints
TODO
## Implementation
#### Parameters
#TODO fix copy pasta from research issue
T -> Item set threshold. If a range length is <= than T, all items are sent. Higher T sends more items which means higher chance of duplicates but reduce the amount of round trips overall.
B -> Partitioning count. When recursively splitting a range, it is split into B sub ranges. Higher B reduce round trips at the cost of computing more fingerprints.
#### Storage
TODO
A local cache of message Ids MUST be maintained when the node is online.
This storage MUST keep Ids ordered at all times.
This storage is critical for the various function of the protocol and should as efficient as possible.
How this storage is implemented however, is outside the scope of this specification.
TODO mention trees vs arrays???
The storage implementation should reflect the Waku context.
Most messages that will be added will be recent and
all removed messages will be older ones.
When differences are found some messages will have to be inserted randomly.
It is expected to be a less likely case than time based insertion and removal.
Last but not least it must be optimized for sequential read
as it is the most often used operation.
#### Range
TODO
We also offset the sync range by 20 seconds in the past.
The actual start of the sync range is T-01:00:20 and the end T-00:00:20
This is to handle the inherent jitters of GossipSub.
In other words, it is the amount of time needed to confirm if a message is missing or not.
#### Interval
TODO
Ad-hoc syncing can be useful in some cases but continuous periodic sync
minimize the differences in messages stored across the network.
Syncing early and often is the best strategy.
The default used in nwaku is 5 minutes interval between sync with a range of 1 hour.
#### Peer Choice
TODO
Peering strategies can lead to inadvertently segregating peers and reduce sampling diversity.
We randomly select peers to sync with for simplicity and robustness.
A good strategy can be devised but we chose not to.
## Attack Vectors ## Attack Vectors
Nodes using `WAKU-SYNC` are fully trusted. Nodes using `WAKU-SYNC` are fully trusted.
@ -92,36 +145,6 @@ Further refinements to the protocol are planned
to reduce the trust level required to operate. to reduce the trust level required to operate.
Notably by verifying messages RLN proof at reception. Notably by verifying messages RLN proof at reception.
## Implementation
The following is not part of the specifications but good to know implementation details.
### Peer Choice
Peering strategies can lead to inadvertently segregating peers and reduce sampling diversity.
We randomly select peers to sync with for simplicity and robustness.
A good strategy can be devised but we chose not to.
### Interval
Ad-hoc syncing can be useful in some cases but continuous periodic sync
minimize the differences in messages stored across the network.
Syncing early and often is the best strategy.
The default used in nwaku is 5 minutes interval between sync with a range of 1 hour.
### Range
We also offset the sync range by 20 seconds in the past.
The actual start of the sync range is T-01:00:20 and the end T-00:00:20
This is to handle the inherent jitters of GossipSub.
In other words, it is the amount of time needed to confirm if a message is missing or not.
### Storage
The storage implementation should reflect the Waku context.
Most messages that will be added will be recent and
all removed messages will be older ones.
When differences are found some messages will have to be inserted randomly.
It is expected to be a less likely case than time based insertion and removal.
Last but not least it must be optimized for sequential read
as it is the most often used operation.
## Copyright ## Copyright
Copyright and related rights waived via Copyright and related rights waived via