mirror of
https://github.com/logos-messaging/specs.git
synced 2026-01-04 07:03:14 +00:00
rephrasing and typos
This commit is contained in:
parent
4732b141dc
commit
8eb1db73bb
@ -9,13 +9,13 @@ contributors:
|
|||||||
|
|
||||||
# Abstract
|
# Abstract
|
||||||
This specification explains `WAKU-SYNC`
|
This specification explains `WAKU-SYNC`
|
||||||
which enables the syncronization of messages between 2 Store nodes.
|
which enables the synchronization of messages between 2 Store nodes.
|
||||||
|
|
||||||
# Specification
|
# Specification
|
||||||
|
|
||||||
Waku Sync consists of 2 protocols; reconciliation and transfer.
|
Waku Sync consists of 2 protocols; reconciliation and transfer.
|
||||||
Reconciliation is the process of finding differences in 2 sets of message hashes.
|
Reconciliation is the process of finding differences in 2 sets of message hashes.
|
||||||
Transfer is then used to bilateraly send messages to the other peer.
|
Transfer is then used to bilaterally send messages to the other peer.
|
||||||
The end goal being that both peers have the same set of hashes and messages.
|
The end goal being that both peers have the same set of hashes and messages.
|
||||||
|
|
||||||
#### Terminology
|
#### Terminology
|
||||||
@ -57,16 +57,16 @@ Every range MUST have one of the following types; skip, fingerprint or item set.
|
|||||||
save network roundtrips.
|
save network roundtrips.
|
||||||
|
|
||||||
#### Range Processing
|
#### Range Processing
|
||||||
Ranges have to be processed differently acording to their types.
|
Ranges have to be processed differently according to their types.
|
||||||
|
|
||||||
- Skip ranges MUST be merged with other consequtive ones if possible.
|
- Skip ranges MUST be merged with other consecutive ones if possible.
|
||||||
- Equal fingerprint ranges MUST become skip ranges.
|
- Equal fingerprint ranges MUST become skip ranges.
|
||||||
- Unequal fingerprint ranges MUST be splitted into smaller ranges. The new type MAY be either fingerprint or item set.
|
- Unequal fingerprint ranges MUST be splitted into smaller ranges. The new type MAY be either fingerprint or item set.
|
||||||
- Unresolved item set ranges MUST be checked for differences and marked resolved.
|
- Unresolved item set ranges MUST be checked for differences and marked resolved.
|
||||||
- Resolved item set ranges MUST be checked for differences and become skip ranges.
|
- Resolved item set ranges MUST be checked for differences and become skip ranges.
|
||||||
|
|
||||||
### Delta Encoding
|
### Delta Encoding
|
||||||
For efficient transmition of timestamps, hashes and ranges. Payloads are delta encoded as follow.
|
For efficient transmission of timestamps, hashes and ranges. Payloads are delta encoded as follow.
|
||||||
|
|
||||||
All ranges to be transmitted MUST be ordered and only upper bounds used.
|
All ranges to be transmitted MUST be ordered and only upper bounds used.
|
||||||
> Inclusive lower bounds can be omitted because they are always
|
> Inclusive lower bounds can be omitted because they are always
|
||||||
@ -79,7 +79,7 @@ This way the receiving peer knows to ignore the range from zero to the start of
|
|||||||
|
|
||||||
Every timestamps after the first MUST be noted as the difference from the previous one.
|
Every timestamps after the first MUST be noted as the difference from the previous one.
|
||||||
If the timestamp is the same, zero MUST be used and the hash MUST be added.
|
If the timestamp is the same, zero MUST be used and the hash MUST be added.
|
||||||
The added hash MUST be trucated up to and including the first differetiating byte.
|
The added hash MUST be truncated up to and including the first differentiating byte.
|
||||||
|
|
||||||
| Timestamp | Hash | Timestamp (encoded) | Hash (encoded)
|
| Timestamp | Hash | Timestamp (encoded) | Hash (encoded)
|
||||||
| - | - | - | -
|
| - | - | - | -
|
||||||
@ -109,13 +109,11 @@ The wire level payload MUST be encoded as follow.
|
|||||||
|
|
||||||
**Libp2p Protocol identifier**: `/vac/waku/transfer/1.0.0`
|
**Libp2p Protocol identifier**: `/vac/waku/transfer/1.0.0`
|
||||||
|
|
||||||
TODO
|
The transfer protocol SHOULD send messages as soon as
|
||||||
|
a difference is found via reconciliation.
|
||||||
should not accept messages from peers not being syncing with.
|
It MUST only accept messages from peers the node is reconciliating with.
|
||||||
|
New message Ids MUST be added to the reconciliation protocol.
|
||||||
should send message as soon as a diff is found.
|
The payload sent MUST follow the wire specification below.
|
||||||
|
|
||||||
in the future verify RLN proof of messages.
|
|
||||||
|
|
||||||
### Wire specification
|
### Wire specification
|
||||||
```protobuf
|
```protobuf
|
||||||
@ -133,16 +131,19 @@ message WakuMessageAndTopic {
|
|||||||
```
|
```
|
||||||
|
|
||||||
# Implementation
|
# Implementation
|
||||||
The flexibitity of the protocol implies that much is left to the implementers.
|
The flexibility of the protocol implies that much is left to the implementers.
|
||||||
What will follow is NOT part of the specification.
|
What will follow is NOT part of the specification.
|
||||||
This section was created to inform implementations.
|
This section was created to inform implementations.
|
||||||
|
|
||||||
#### Parameters
|
#### Parameters
|
||||||
#TODO fix copy pasta from research issue
|
Two useful parameters to add to your implementation are partitioning count and the item set threshold.
|
||||||
|
|
||||||
T -> Item set threshold. If a range length is <= than T, all items are sent. Higher T sends more items which means higher chance of duplicates but reduce the amount of round trips overall.
|
The partitioning count is the number of time a range is splitted.
|
||||||
|
Higher value reduce round trips at the cost of computing more fingerprints.
|
||||||
|
|
||||||
B -> Partitioning count. When recursively splitting a range, it is split into B sub ranges. Higher B reduce round trips at the cost of computing more fingerprints.
|
The threshold for which item sets are sent instead of fingerprints.
|
||||||
|
Higher value sends more items which means higher chance of duplicates but
|
||||||
|
reduce the amount of round trips overall.
|
||||||
|
|
||||||
#### Storage
|
#### Storage
|
||||||
The storage implementation should reflect the context.
|
The storage implementation should reflect the context.
|
||||||
@ -153,31 +154,24 @@ It is expected to be a less likely case than time based insertion and removal.
|
|||||||
Last but not least it must be optimized for fingerprinting
|
Last but not least it must be optimized for fingerprinting
|
||||||
as it is the most often used operation.
|
as it is the most often used operation.
|
||||||
|
|
||||||
TODO mention trees vs arrays???
|
|
||||||
|
|
||||||
#### Sync Window
|
|
||||||
TODO rephrase
|
|
||||||
|
|
||||||
We also offset the sync window by 20 seconds in the past.
|
|
||||||
The actual start of the sync range is T-01:00:20 and the end T-00:00:20
|
|
||||||
This is to handle the inherent jitters of GossipSub.
|
|
||||||
In other words, it is the amount of time needed to confirm if a message is missing or not.
|
|
||||||
|
|
||||||
#### Sync Interval
|
#### Sync Interval
|
||||||
TODO rephrase
|
|
||||||
|
|
||||||
Ad-hoc syncing can be useful in some cases but continuous periodic sync
|
Ad-hoc syncing can be useful in some cases but continuous periodic sync
|
||||||
minimize the differences in messages stored across the network.
|
minimize the differences in messages stored across the network.
|
||||||
Syncing early and often is the best strategy.
|
Syncing early and often is the best strategy.
|
||||||
The default used in nwaku is 5 minutes interval between sync with a range of 1 hour.
|
The default used in Nwaku is 5 minutes interval between sync with a range of 1 hour.
|
||||||
|
|
||||||
|
#### Sync Window
|
||||||
|
By default we offset the sync window by 20 seconds in the past.
|
||||||
|
The actual start of the sync range is T-01:00:20 and the end T-00:00:20 in most cases.
|
||||||
|
This is to handle the inherent jitters of GossipSub.
|
||||||
|
In other words, it is the amount of time needed to confirm if a message is missing or not.
|
||||||
|
|
||||||
#### Peer Choice
|
#### Peer Choice
|
||||||
TODO rephrase
|
Wrong peering strategies can lead to inadvertently segregating peers and
|
||||||
|
reduce sampling diversity.
|
||||||
|
Nwaku randomly select peers to sync with for simplicity and robustness.
|
||||||
|
|
||||||
Peering strategies can lead to inadvertently segregating peers and reduce sampling diversity.
|
Good strategies can be devised but we chose not to.
|
||||||
We randomly select peers to sync with for simplicity and robustness.
|
|
||||||
|
|
||||||
A good strategy can be devised but we chose not to.
|
|
||||||
|
|
||||||
## Attack Vectors
|
## Attack Vectors
|
||||||
Nodes using `WAKU-SYNC` are fully trusted.
|
Nodes using `WAKU-SYNC` are fully trusted.
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user