nim-chat-sdk/README.segmentation.md

Message Segmentation

Split large payloads into Waku-sized segments with optional Reed–Solomon parity for robust reconstruction. Inbound segmented messages are reassembled and persisted until complete.

Features

• Segmentation: Outbound payloads over the configured size are split into segments; smaller payloads pass through unchanged.
• Reconstruction: Inbound segments can be processed out of order and reassembled into the original message.
• Erasure coding: Adds parity segments at 12.5% of data segments (≤ SegmentsReedsolomonMaxCount), enabling recovery with ~87.5% of segments.
• Persistence: Segments and completion state are stored (SQLite) and survive restarts.
• API simplicity: Main parameter is the maximum segment size; inject your persistence object.

Essential API

• Types: SegmentationHander, Chunk, Message
• Outbound: segmentMessage(handler, chunk: Chunk): Result[seq[Chunk], string]
• Inbound: handleSegmentationLayer(handler, message: var Message): Result[void, string]
• Helpers: isParityMessage(segment: SegmentMessage): bool

Notes

• Parity rate is 12.5% (SegmentsParityRate = 0.125).

Wire format

Segments are protobuf-encoded (segment_message.proto):

Tests

See tests/test_segmentation.nim for examples covering in-order/out-of-order and parity recovery paths.