SDS-R is an optional extension to the Scalable Data Sync (SDS) protocol that enables collaborative repair of missing messages within a limited time window. It's designed to work over Waku and assumes participants are already in a secure channel.
## Core Concept
When a participant detects missing messages (via causal dependencies), it waits a random backoff period before requesting repairs. Other participants who have the missing message wait their own random backoff before responding. The protocol uses clever timing and grouping to ensure typically only one request and one response per missing message.
broadcast(message) # Rebroadcast the full original message
remove_from_incoming_buffer(entry)
```
### Periodic Sync Messages with SDS-R
When sending periodic sync messages:
1. Check if there are eligible entries in outgoing repair request buffer
2. If yes, send the sync message WITH repair_request field populated
3. Unlike base SDS, don't suppress sync message even if others recently sent one
---
## Implementation Notes and Edge Cases
### Hash Function
**CRITICAL**: The spec doesn't specify which hash function to use. Recommend:
- Use SHA256 for cryptographic properties
- Convert to integer for modulo operations: `int(hash_bytes[:8], byteorder='big')`
- Must be consistent across all participants
### Participant ID Format
- Must support XOR operation for distance calculation
- Recommend using numeric IDs or convert string IDs to integers
- Must be globally unique within the channel
### Memory Management
1.**Buffer limits**: Implement max size for repair buffers (suggest 1000 entries)
2.**Eviction policy**: Remove oldest T_req/T_resp when at capacity
3.**History retention**: Only keep messages for T_max duration
4.**Response group optimization**: Only cache full messages if you're likely to be in response group
### Edge Cases to Handle
1.**Duplicate repair requests**: Use Set semantics, only track once
2.**Expired repairs**: If T_req > current_time + T_max, remove from buffer
3.**Non-numeric participant IDs**: Hash to integer for XOR operations
4.**Missing sender_id**: Cannot participate in repair for that message
5.**Circular dependencies**: Set maximum recursion depth for dependency resolution
### Typo to Fix
The spec has "Perdiodic" on line 461 - should be "Periodic"
---
## Testing Scenarios
1.**Single missing message**: Verify only one repair request and response
2.**Cascade recovery**: Missing message A depends on missing message B
3.**Original sender offline**: Verify next closest participant responds
4.**Response group isolation**: Verify only in-group participants respond
5.**Buffer overflow**: Test eviction policies
6.**Network partition**: Test behavior when repair window expires
---
## Integration with Base SDS
### Modified State from Base SDS
- Local history stores full Messages, not just IDs
- Additional buffers for repair tracking
- Sender_id must be preserved in HistoryEntry
### Unchanged from Base SDS
- Lamport timestamp management
- Bloom filter operations
- Causal dependency checking
- Message delivery and conflict resolution
---
## Performance Recommendations
1. Use priority queues for T_req/T_resp ordered buffers
2. Index local history by message_id for O(1) lookup
3. Batch repair requests in single message (up to 3)
4. Cache response group calculation results
5. Implement exponential backoff in future version (noted as TODO in spec)
---
## Security Assumptions
- Operating within secure channel (via Waku)
- All participants are authenticated
- Rate limiting via Waku RLN-RELAY
- No additional authentication needed for repairs
- Trust all repair requests from channel members
This implementation guide should be sufficient to implement SDS-R without access to the original specification. The key insight is that SDS-R elegantly uses timing and randomization to coordinate distributed repair without central coordination or excessive network traffic.
