mirror of
https://github.com/logos-messaging/specs.git
synced 2026-05-25 22:39:28 +00:00
address comments, improvements
This commit is contained in:
parent
4f3c1d9921
commit
4af5b4ec56
@ -17,8 +17,7 @@ Messages whose payload size is **≤ `segmentSize`** are sent unmodified.
|
|||||||
## Motivation
|
## Motivation
|
||||||
|
|
||||||
Waku Relay deployments typically propagate envelopes up to **150 KB** as per [64/WAKU2-NETWORK - Message](https://rfc.vac.dev/waku/standards/core/64/network#message-size).
|
Waku Relay deployments typically propagate envelopes up to **150 KB** as per [64/WAKU2-NETWORK - Message](https://rfc.vac.dev/waku/standards/core/64/network#message-size).
|
||||||
To support larger application payloads,
|
To support larger application payloads, a segmentation layer is required.
|
||||||
a segmentation layer is required.
|
|
||||||
This specification enables larger messages by partitioning them into multiple envelopes and reconstructing them at the receiver.
|
This specification enables larger messages by partitioning them into multiple envelopes and reconstructing them at the receiver.
|
||||||
Erasure-coded parity segments provide resilience against partial loss or reordering.
|
Erasure-coded parity segments provide resilience against partial loss or reordering.
|
||||||
|
|
||||||
@ -29,7 +28,6 @@ Erasure-coded parity segments provide resilience against partial loss or reorder
|
|||||||
- **parity segment**: an erasure-coded segment derived from the set of data segments.
|
- **parity segment**: an erasure-coded segment derived from the set of data segments.
|
||||||
- **segment message**: a wire-message whose `payload` field carries a serialized `SegmentMessageProto`.
|
- **segment message**: a wire-message whose `payload` field carries a serialized `SegmentMessageProto`.
|
||||||
- **`segmentSize`**: configured maximum size in bytes of each data segment's `payload` chunk (before protobuf serialization).
|
- **`segmentSize`**: configured maximum size in bytes of each data segment's `payload` chunk (before protobuf serialization).
|
||||||
- **sender public key**: the origin identifier used for indexing persistence.
|
|
||||||
|
|
||||||
The key words **"MUST"**, **"MUST NOT"**, **"REQUIRED"**, **"SHALL"**, **"SHALL NOT"**, **"SHOULD"**, **"SHOULD NOT"**, **"RECOMMENDED"**, **"NOT RECOMMENDED"**, **"MAY"**, and **"OPTIONAL"** in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
|
The key words **"MUST"**, **"MUST NOT"**, **"REQUIRED"**, **"SHALL"**, **"SHALL NOT"**, **"SHOULD"**, **"SHOULD NOT"**, **"RECOMMENDED"**, **"NOT RECOMMENDED"**, **"MAY"**, and **"OPTIONAL"** in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||||
|
|
||||||
@ -46,14 +44,14 @@ message SegmentMessageProto {
|
|||||||
|
|
||||||
// Data segment indexing
|
// Data segment indexing
|
||||||
uint32 index = 2; // zero-based sequence number; valid only if segments_count > 0
|
uint32 index = 2; // zero-based sequence number; valid only if segments_count > 0
|
||||||
uint32 segment_count = 3; // number of data segments (>= 2)
|
uint32 segments_count = 3; // number of data segments
|
||||||
|
|
||||||
// Segment payload (data or parity shard)
|
// Segment payload (data or parity shard)
|
||||||
bytes payload = 4;
|
bytes payload = 4;
|
||||||
|
|
||||||
// Parity segment indexing (used if segments_count == 0)
|
// Parity segment indexing (used if segments_count == 0)
|
||||||
uint32 parity_segment_index = 5; // zero-based sequence number for parity segments
|
uint32 parity_segment_index = 5; // zero-based sequence number for parity segments
|
||||||
uint32 parity_segments_count = 6; // number of parity segments (> 0)
|
uint32 parity_segments_count = 6; // number of parity segments
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -61,12 +59,12 @@ message SegmentMessageProto {
|
|||||||
|
|
||||||
- `entire_message_hash`: A 32-byte Keccak256 hash of the original complete payload, used to identify which segments belong together and verify reconstruction integrity.
|
- `entire_message_hash`: A 32-byte Keccak256 hash of the original complete payload, used to identify which segments belong together and verify reconstruction integrity.
|
||||||
- `index`: Zero-based sequence number identifying this data segment's position (0, 1, 2, ..., segments_count - 1).
|
- `index`: Zero-based sequence number identifying this data segment's position (0, 1, 2, ..., segments_count - 1).
|
||||||
- `segment_count`: Total number of data segments the original message was split into.
|
- `segments_count`: Total number of data segments the original message was split into.
|
||||||
- `payload`: The actual chunk of data or parity information for this segment.
|
- `payload`: The actual chunk of data or parity information for this segment.
|
||||||
- `parity_segment_index`: Zero-based sequence number for parity segments.
|
- `parity_segment_index`: Zero-based sequence number for parity segments.
|
||||||
- `parity_segments_count`: Total number of parity segments generated.
|
- `parity_segments_count`: Total number of parity segments generated.
|
||||||
|
|
||||||
A message is either a **data segment** (when `segment_count > 0`) or a **parity segment** (when `segment_count == 0`).
|
A message is either a **data segment** (when `segments_count > 0`) or a **parity segment** (when `segments_count == 0`).
|
||||||
|
|
||||||
### Validation
|
### Validation
|
||||||
|
|
||||||
@ -127,7 +125,7 @@ The reference implementation uses **nim-leopard** (Leopard-RS) with a maximum of
|
|||||||
|
|
||||||
### Storage / Persistence
|
### Storage / Persistence
|
||||||
|
|
||||||
Segments **MAY** be persisted (e.g., SQLite) and indexed by `entire_message_hash` and sender public key.
|
Segments **MAY** be persisted (e.g., SQLite) and indexed by `entire_message_hash` and by sender. Sender MAY be authenticated, this is out of scope of this spec.
|
||||||
Implementations **SHOULD** support:
|
Implementations **SHOULD** support:
|
||||||
|
|
||||||
- Duplicate detection and idempotent saves
|
- Duplicate detection and idempotent saves
|
||||||
@ -137,15 +135,12 @@ Implementations **SHOULD** support:
|
|||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
**Required parameters:**
|
- `segmentSize` — maximum size in bytes of each data segment's payload chunk (before protobuf serialization).
|
||||||
|
**REQUIRED** parameter, configurable by the client.
|
||||||
- `segmentSize` — **REQUIRED** configurable parameter;
|
- `parityRate` — fraction of parity shards relative to data shards.
|
||||||
maximum size in bytes of each data segment's payload chunk (before protobuf serialization).
|
Configurable by the client. Defaults to **0.125** (12.5%).
|
||||||
|
- `maxTotalSegments` — maximum number of total shards (data + parity) per message.
|
||||||
**Fixed parameters:**
|
Implementation-specific parameter, fixed. The reference implementation uses **256**.
|
||||||
|
|
||||||
- `parityRate` — fixed at **0.125** (12.5%)
|
|
||||||
- `maxTotalSegments` — **256**
|
|
||||||
|
|
||||||
**Reconstruction capability:**
|
**Reconstruction capability:**
|
||||||
With the predefined parity rate,
|
With the predefined parity rate,
|
||||||
@ -154,6 +149,8 @@ reconstruction is possible if **all data segments** are received or if **any com
|
|||||||
**API simplicity:**
|
**API simplicity:**
|
||||||
Libraries **SHOULD** require only `segmentSize` from the application for normal operation.
|
Libraries **SHOULD** require only `segmentSize` from the application for normal operation.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Support
|
### Support
|
||||||
|
|
||||||
- **Language / Package:** Nim;
|
- **Language / Package:** Nim;
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user