logos-messaging/specs
1
0
Fork 0
mirror of https://github.com/logos-messaging/specs.git synced 2026-05-23 21:39:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of github.com:waku-org/specs into jazzz/private

Browse Source
This commit is contained in:
Jazz Turner-Baggs 2026-05-22 16:39:30 -07:00
commit d32d10b99d
No known key found for this signature in database
14 changed files with 2537 additions and 373 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

229
.wordlist.txt
View File

@ -1,51 +1,278 @@
ABIs
acknowledgementTimeoutMs
addListener
aea
AEAD
ALLOC
api
AsyncAPI
autosharding
AutoShardingConfig
ba
backend
Backend
backends
Baggs
Bande
BCP
bool
Bradner
camelCase
causalHistorySize
cd
centric
Changelog
channelConfig
channelId
chatintro
ciphertext
CLI
closeChannel
config
confusable
ConnectionStatus
ConnectionStatusEvent
ContentFrame
contentTopic
contentTopics
createNode
createReliableChannel
creativecommons
cryptographic
cryptographically
da
danielkaiser
dataSegments
decrypt
Decrypt
decrypted
Decrypts
DefaultAutoShardingConfig
DefaultMessageValidation
DefaultNetworkingConfig
DefaultRateLimitConfig
DefaultSdsConfig
DefaultSegmentationConfig
Deployability
dev
df
dh
DHT
Diffie
discv
DISCV
DoS
ec
enableReedSolomon
encodings
encryptionKey
enrtree
enum
epochPeriodSec
epochSizeMs
eth
Eth
ETH
EventEmitter
EventMessageReceived
EventMessageSendError
EventMessageSent
EventReliableMessageSent
EventSource
eventType
fb
fBF
fc
fd
Folgueira
getAvailableConfigs
getAvailableNodeInfoIds
getAvailableNodeInfoItems
getMerkleRoot
getNodeInfo
github
GITHUB
gossipsub
GossipSub
Groupchats
HealthConnectionStatusEvent
healthEvents
HealthEvents
Hellman
historyBackend
HKDF
https
iana
IANA
IDL
IEncryption
implementers
implementor
implementors
Implementors
Inclusivity
Init
IntroBundle
IPersistence
ipv
iterable
Jazzz
JSON
KDF
keccak
Keccak
KiB
Kozlov
lamport
liblogosdelivery
libp
libp2p
lifecycle
Lightpush
LIGHTPUSH
maxRetransmissions
maxTotalSegments
md
MessageDeliveredEvent
MessageDeliveryErrorEvent
messageEnvelope
MessageEnvelope
MessageErrorEvent
messageEvents
MessageEvents
MessageHash
MessageId
MessagePropagatedEvent
MessageReceivedEvent
MessageSendErrorEvent
MessageSendPropagatedEvent
MessageSentEvent
MessageValidation
MiB
multiaddr
namespace
NetworkConfig
NetworkingConfig
nim
Nim
nodeConfig
NodeConfig
nodeInfoId
nonces
num
observability
Oleksandr
onEvent
onMessageDelivered
onMessageReceived
onMessageSendError
onMessageSent
OpenAPI
openChannel
parityRate
Parsers
PartiallyConnected
PascalCase
Patryk
Pax
permissionless
plaintext
pluggable
PoV
Prathi
pre
Prem
proto
protobuf
ProtocolsConfig
pubkey
publickey
pubsub
rateLimitConfig
RateLimitConfig
Raya
Raya's
ReliableChannel
ReliableChannelConfig
ReliableEnvelope
ReliableIrretrievableMessageEvent
ReliableMessageAcknowledgedEvent
ReliableMessageEvents
ReliableMessageReceivedEvent
ReliableMessageSendErrorEvent
ReliableMessageSentEvent
ReliableSendId
ReliableSyncStatusEvent
Req
requestId
RequestId
responder
retransmission
Retransmission
retransmit
retransmitted
retransmitting
rfc
RFC
rln
RLN
RlnConfig
Royer
rpc
RPC
Saro
Scalable
sdk
SDK
sds
SDS
SDS'ed
sdsConfig
SdsConfig
segmentationConfig
SegmentationConfig
SegmentMessageProto
segmentSize
segmentSizeBytes
senderId
SHA
sharding
SHARDING
Sirotin
sqlite
subnets
SubscriptionError
syncStatus
SyncStatus
SyncStatusDetail
TBD
tcp
TCP
th
TheWakuNetworkMessageValidation
TheWakuNetworkPreset
TODO
TWN
udp
UDP
uint
uint32
unencrypted
Unencrypted
unvalidated
url
UTF
UUID
UX
waku
Waku
WAKU
Waku's
WakuNode
www
ZXCV
xB
XEdDSA
yaml
YAML
yml
ZXCV

1
README.md
View File

@ -50,6 +50,7 @@ This repository contains specifications for the Waku suite of protocols.
|[TOR-PUSH](standards/application/tor-push.md)| Waku Tor Push |
|[RLN-KEYSTORE](standards/application/rln-keystore.md)| Waku RLN Keystore JSON schema |
|[P2P-RELIABILITY](standards/application/p2p-reliability.md)| Waku P2P Reliability |
|[WAKU-API](standards/application/waku-api.md)| Waku API |
### Informational

104
informational/chat_cast.md Normal file
View File

@ -0,0 +1,104 @@
---
title: CHAT-CAST
name: Roles and Entities Used in Chat Protocol Documentation
status: raw
category: Informational
tags: [chat/informational]
editor: Jazzz <jazz@status.im>
contributors:
---
## Abstract
This document defines a reusable cast of characters to be used in chat protocol documentation and related supporting materials.
The goal is to improve clarity and consistency when describing protocol roles, actors, and message flows.
## Background
When documenting applications and protocols, it is often beneficial to define a consistent set of characters representing common roles.
A shared cast allows authors to convey meaning and intent quickly to readers.
Readers are not required to understand these meanings, however consistent usage can make comprehension faster to achieve.
This approach is well established in cryptographic literature, where `Alice` and `Bob` are commonly used to describe participants in key exchange protocols.
Within that context, Alice typically initiates the exchange and Bob responds.
Readers familiar with this convention can quickly understand protocol flows without additional explanation.
In messaging and communication protocols, a similar approach can be helpful, particularly when describing multiple actors and roles required for correct protocol operation.
However, reusing `Alice` and `Bob` in these contexts can introduce ambiguity:
- In cryptography, Alice is the initiator of a key exchange, but in a communication protocol the initiator role may vary by sub-protocol or phase.
- Complex, multi-step systems may embed multiple cryptographic and application-level processes, each with their own initiator and responder.
- The use of Alice and Bob implicitly frames the discussion as cryptographic, which may be misleading when describing application-level behavior such as message encoding, routing, or reliability.
For these reasons, when documenting communication protocols that integrate multiple roles and procedures, it is preferable to use a context-specific cast of characters designed for that domain.
## Guidelines
### Use of Alice and Bob
`Alice` and `Bob` SHOULD be used when describing novel cryptographic constructions or key exchange mechanisms that are not embedded within higher-layer communication protocols.
These names are widely understood in cryptographic contexts, and introducing alternatives would reduce clarity.
Communication protocols may incorporate cryptographic components, but they are not themselves cryptographic key exchanges.
When documenting application-centric or protocol-level processes, the cast defined in this document SHOULD be used instead.
This separation establishes clear contextual boundaries and prepares the reader to reason about different layers independently.
### Standalone Documents
Knowledge of this cast MUST NOT be a requirement to understand a given document.
Documents MUST be fully standalone and understandable to first-time readers.
### Consistency
Use of the cast is optional.
Characters SHOULD only be used when their presence improves understanding.
Using characters in the wrong context can negatively impact comprehension by implying incorrect information.
It is always acceptable to use other identifiers.
## Character List
The following characters are defined for use throughout the documentation of chat protocols. Documentation and examples focus on a portion of a real clients operation for simplicity. Using the character who corresponds to the role or perspective being highlighted, can help convey this information to readers.
**Saro**
Sender ::
Saro is the participant who sends the first message within a given time window or protocol context.
Saro MAY be the party who initiates a conversation, or simply the first participant to act relative to a defined starting reference.
**Raya**
Recipient ::
Raya is the participant who receives the first message sent by Saro.
After the initial exchange, Raya MAY send messages and behave as a regular participant in the conversation.
When documenting message receipt or processing, Raya’s perspective SHOULD be used.
**Pax**
Participant ::
Pax represents an additional member of a conversation, typically in a group context.
Pax is often used when the specific identity or perspective of the participant is not relevant to the discussion.
## Decision Criteria
The following criteria SHOULD be applied when considering the introduction of new character names or roles.
### Clarity
Names without strong pre-existing associations or implied behavior SHOULD be preferred where possible.
### Brevity
Short, easily distinguishable names SHOULD be preferred, provided they do not reduce clarity.
### Inclusivity
The cast of characters SHOULD be diverse, culturally neutral, and avoid reinforcing stereotypes.
### Mnemonic Naming
Where possible the characters name should hint at their role in order to make them easier to remember.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
[A global cast of characters for cryptography](https://github.com/jhaaaa/alix-and-bo)

98
informational/chatdefs.md Normal file
View File

@ -0,0 +1,98 @@
---
title: CHAT-DEFINITIONS
name: Shared definitions for Chat Protocols
category: Informational
tags: definitions, terminology, reference
editor:
contributors:
---
## Abstract
This specification establishes a common vocabulary and standardized definitions for terms used across chat protocol specifications. It serves as a normative reference to ensure consistent interpretation of terminology across application level specifications.
## Background / Rationale / Motivation
Chat protocol specifications often employ similar terminology with subtle but important semantic differences. The absence of standardized definitions leads to ambiguity in specification interpretation and implementation inconsistencies across different protocol variants.
Establishing a shared definitions document provides several benefits:
1. **Semantic Consistency**: Ensures uniform interpretation of common terms across multiple specifications
2. **Implementation Clarity**: Reduces ambiguity for protocol implementers
3. **Specification Quality**: Enables more precise and concise specification language through reference to established definitions
4. **Ecosystem Coherence**: Promotes consistent understanding within the chat protocol ecosystem.
This specification provides normative definitions that other chat protocol specifications MAY reference to establish precise semantic meaning for common terminology.
## Theory / Semantics
### Definition Categories
Terms are organized into the following categories for clarity and ease of reference:
- **Roles**: Defined entity types that determine how a participant behaves within a communication protocol.
- **Message Types**: Classifications and categories of protocol messages
- **Transports**: Abstract methods of transmitting payloads
-**Software Entities**: Distinct software-defined actors or components that participate in the operation of chat protocols.
## Definitions
### Roles
**Sender**: A client which is pushing a payload on to the network, to one or more recipients.
**Recipient**: A client which is the intended receiver of a payload. In a group context there maybe multiple recipients
**Participant**: A generic term for the rightful members of a conversation. Senders and Recipients are roles that participants can hold.
### Message Types
The term "message" often has multiple meanings depending on context. The following definitions are used to disambiguate between different abstraction levels.
**Content**: An opaque byte sequence whose meaning is defined solely by the Application. The chat protocol layer neither interprets nor validates Content structure.
**Frame**: A structured protocol message exchanged between Clients. Frames are typed data structures that carry protocol meaning — they are how clients coordinate state and exchange information. Some Frames may carry Content as an opaque field. Once serialized for transport they become Payloads.
**Payload**: The serialized binary representation of a Frame, treated by the transport layer as an opaque byte sequence with no chat-layer semantics.
Other specific message types include:
**Content Type**: A definition of the structure and encoding of a Content instance, interpreted solely by the Application.
**Delivery Acknowledgement**: A notification from a receiving client to sender that their message was successfully received. While similar to a read-receipt, delivery acknowledgements differ in that the acknowledgement originates based on the client, where read-receipts are fired when they are displayed to a user.
**Invite**: A frame used to initialize a new conversation. Invites notify a client that someone wants to communicate with them, and provides the required information to do so.
### Transports
**Out-of-Band**: The transfer of information using a channel separate from the defined chat protocol. Data sent Out-of-Band is transmitted using an undefined mechanism. This is used when protocols requires information to be shared with another entity, but it does not describe how that should occur. The responsibility to define how this occurs is the implementer or other protocols in the suite.
### Software Entities
**Client**: A software component that manages Conversations and exposes messaging capabilities to Applications. The Client acts as the interface between the Application and the underlying protocol.
**Application**: Software that integrates with a Client in order to send and receive content. Applications are responsible for displaying content and controlling what content gets sent.
**Conversation**: An instance of a chat protocol between a set of participants. Conversations are responsible for protocol operations including encryption, key management, and frame generation.
## Wire Format Specification / Syntax
This specification does not define wire format elements. All definitions are semantic and apply to the interpretation of terms used in other specifications that do define wire formats.
## Security/Privacy Considerations
This specification defines terminology only and does not introduce security or privacy considerations beyond those present in specifications that reference these definitions.
The definitions provided in this specification do not alter the security or privacy properties of implementations that adopt the defined terminology.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

249
standards/application/OpChan.md Normal file
View File

@ -0,0 +1,249 @@
---
title: OP-CHAN
name: OpChan Decentralized Forum
status: raw
category: Standards Track
tags: waku
editor:
contributors:
- Jimmy Debe <jimmy@status.im>
---
## Abstract
This document specifies the architecture of OpChan,
a decentralized forum application.
The specification is transport-agnostic,
with Waku as the reference delivery mechanism.
## Background
In a decentralized forum, content is hosted on multiple nodes,
making it difficult for a user's post to be censored.
Users own their post data, so data cannot be removed by a third party,
and forum boards will not rely on moderators remaining active.
OpChan clients distribute content through a peer-to-peer network
rather than relying on centralized server storage.
OpChan supports ephemeral anonymous sessions
using a locally generated [ED25519][ed25519] key pair for identity and signing.
Additionally, OpChan supports wallet-backed identities and identity key delegation.
## Terminology
- Channel: A discussion board or channel that hosts posts and moderation controls.
- Post: User content created within a forum.
- Comment: A reply to a `Post` or other `Comment` (threaded discussion).
- Participant: Any user able to publish or consume messages (anonymous or
wallet-backed).
- Anonymous session: A client-generated ed25519 keypair used as
identity for a user without a wallet identity.
## Specification
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in [RFC 2119][rfc2119].
OpChan distributes forum content amongst peers using a publish-subscribe messaging layer.
All messages are cryptographically signed using [ED25519][ed25519] keys generated by the client locally.
OpChan supports two types of messages,
content and control messages.
Content messages are user-generated content on the forum.
The message types include the following:
- Channel: Includes the metadata information like name, description, and admins of a forum feed.
- Post: User content created within a channel.
- Comment: Users reply to a post or another comment.
- Vote: To cast upvote or downvote for a post or comment.
- User: Includes username, delegation proofs, and identities.
See the [Identity](#identity) section for more information.
- Bookmark: A user bookmarking a post or comment.
Control messages consist of the user activity/interactions on the forum.
This includes the management of the current forum state,
permissions, and moderations.
The message types include the following:
- Create Channel: The initial event of creating a new channel within the forum.
- Delegation Events: Channel admins granting or revoking channel rights.
- Moderation Events: Channel admins are able to hide, remove, pin/unpin a post or comment,
promote new admins, and change the ownership of a channel.
### Message Format
Each channel is assigned a unique topic identifier that clients MUST subscribe to
in order to discover messages from that channel.
All messages MUST include the following envelope fields:
``` js
{
"id": "string",
"type": "CHANNEL_CREATED | POST_CREATED | COMMENT_CREATED | VOTE | BOOKMARK | MODERATION | DELEGATION",
"timestamp": "uint64"
"author": "string" // author public key or wallet address
"signature": "bytes" // ed25519 signature
"delegationProof": "bytes" // optional wallet signature authorizing the browser key
"body": object // The message content
}
```
Every message SHOULD be signed using `signature` owned by the publishing user.
Clients SHOULD verify the signature against the `author` public key and,
when present, verify `delegationProof`.
Signing Flow:
1. Serialize `body` fields in stable key order.
2. Construct signing bytes with: `type`, `timestamp`, `author` and `body`.
3. Sign with the user's cryptographic keys, `signature`, for the session.
### Identity
There are two types of identities for users:
anonymous session and wallet delegation.
A wallet delegation MAY be an ENS (Ethereum Name Service) verified user.
An anonymous session generates an [ED25519][ed25519] keypair locally with the client.
The key is used to sign all messages and
a username MAY be attached to the post or comments.
An anonymous user SHOULD NOT be granted an admin role,
create moderation events, or create a channel.
A wallet delegation is a user blockchain wallet's private key used to sign a message,
which is the `delegationProof`.
The `delegationProof` SHOULD be a short-lived message,
RECOMMENDED a few minutes to a few hours.
Once a `delegationProof` is generated,
the client SHOULD be able to sign messages,
without the need for repeated wallet prompts requesting to sign.
#### Delegation Flow
1. The client generates a new `browserKey`, which is an Ed25519 keypair.
2. The client generates a delegation request to authorize `browserKey` to sign.
3. The user's wallet signs the delegation request and
returns a `delegationProof` with an expiration timestamp, `expiry`.
4. The client stores the `delegationProof`, `browserKey`,
and `expiry`.
A `delegationProof` could become revoked by the wallet owner or
after the `expiry` time.
If a wallet delegation is revoked,
clients SHOULD ignore subsequent messages from the revoked delegation key.
### Moderation & Permissions
A post MAY have moderation message types assigned by the channel admin.
The moderation types include:
- `HIDE`: Hide a post or comment from the channel feed.
- `REMOVE`: Permanently remove a post or comment.
- `PIN`: Pin a post to the top of a channel feed or pin a comment to the top of a thread.
- `UNPIN`: Remove a `PIN` from a post feed or comment thread.
- `CHANGE_OWNERSHIP`: Change the `author` of a post or comment.
Moderation messages MUST be signed by an admin,
who is recognized as the `author` of the channel.
Clients SHOULD validate the admin before applying moderation events locally.
#### User Flagging
Users MAY flag content for review by moderators.
A `FLAG` is user-initiated and distinct from admin moderation actions.
Flagged content SHOULD be queued for moderator review
but does not automatically result in content removal.
### Relevance Score
A post can gain better visibility on the forum channel and
the forum's search through the content relevance score.
Clients with verified wallet identities MUST be favored over an anonymous session identity.
There are a few RECOMMENDED areas that collect points when calculating the relevance score of a post:
Basic points include the activities that each user is able to engage in.
- A channel has a score value of 15
- Each post within a channel has a score value of 10
- A base value if comments are present within a post boosts the relevance score by a value of 5
Engagement points include the different user activities for each post or comment.
- Each wallet delegation upvote adds a score value of 1.
- Each individual comment adds a score value of 0.5.
- The total number of posts multiplied by 0.5.
The total number of upvotes multiplied by 0.1.
These two values are added together.
For identity verification points,
participants of posts or comments that use wallet-based identities,
including an optional ENS,
the score is boosted over the anonymous identities.
If a participant has a verified ENS and a verified connected wallet,
only the ENS multiplier SHOULD be applied.
- Participants who have a verified ENS name gain a value multiplier of 1.25(25%).
- For wallet connect participant the multiplier is 1.1(10%).
- For verified upvote participants the multiplier is 0.1.
- For verified comment participants the multiplier is 0.05.
There is a time decay that reduces the relevance score over time.
The older a post or comment was made the lower its score.
$$
\text{timeDecayMultiplier} = e^{-\lambda \cdot \text{daysOld}}
$$
Where $$\( -\lambda \)$$ is the time-decay rate per day.
There SHOULD be a moderation penalty that reduces the score when a post or
comment is moderated with a value of 0.5(50%).
This penalty is applied once a post is `HIDDEN`, `REMOVED` or flagged by users to be reviewed by a moderator.
$$
\text{moderationPenalty} = \begin{cases} 0.5, & \text{if moderated} \\
1, & \text{otherwise} \end{cases}
$$
Below is the final relevance score based on the RECOMMENDED points above:
$$
\text{Total RelevanceScore} = (\text{basic} + \text{engagement} + \text{verifiedUpvote})
\cdot (1 + \text{verifyIdentity}) \cdot \text{timeDecay} \cdot \text{moderationPenalty}
$$
## Waku as Delivery Mechanism
This section describes how OpChan MAY use the Waku protocol as the delivery mechanism.
OpChan clients MAY use the [10/WAKU2][waku2] network for the distribution of forum content amongst peers.
The messages are [14/WAKU-MESSAGE][waku-message] objects.
Users SHOULD use the [19/WAKU2-LIGHTPUSH][waku-lightpush] protocol
to send messages to Waku nodes storing the forum's content.
Message routing and discovery are handled by [23/WAKU2-TOPICS][waku-topics].
Each channel is assigned a `content_topic` that clients MUST subscribe to
in order to discover messages from that channel.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [RFC 2119][rfc2119]
- [ED25519][ed25519]
- [10/WAKU2][waku2]
- [14/WAKU-MESSAGE][waku-message]
- [19/WAKU2-LIGHTPUSH][waku-lightpush]
- [23/WAKU2-TOPICS][waku-topics]
[rfc2119]: https://www.ietf.org/rfc/rfc2119.txt
[ed25519]: https://datatracker.ietf.org/doc/html/rfc8032
[waku2]: https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md
[waku-message]: https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md
[waku-lightpush]: https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md
[waku-topics]: https://github.com/vacp2p/rfc-index/blob/main/waku/informational/23/topics.md

137
standards/application/chat-framework.md Normal file
View File

@ -0,0 +1,137 @@
---
title: CHAT-FRAMEWORK
name: A modular framework for defining chat protocols
category: Information
status: raw
tags: chat
editor: Jazz Alyxzander<jazz@status.im>
contributors:
---
# Abstract
This specification defines a modular communication protocol framework for describing chat protocols. It introduces abstraction boundaries, and a component model for describing chat protocol functionality.
# Background / Rationale / Motivation
Chat protocols specifications can be long and dense documents. To fully describe a chat protocol there are many layers and operations which are required to be documented. This includes payloads, message transport, encryption as well as user level features such as account registration, typing indicators, content formatting.
With the vast amount of information required to maintain compatibility between applications - protocol documentation is either comprehensive which leads to large monolithic specifications or lacking the required details for interop between implementors. A suitable solution would provide both the specificity while also remaining lean and focused.
# Theory / Semantics
This specification defines an abstract framework for building a chat protocol. Its purpose is to name the distinct components/phases, and define modular boundaries between them to promote reuse. The end result is that a chat protocol implementation can be described by listing its approach to 5 things.
The lifecycle of a protocol instance is divided into three phases:
- **Discovery:** How does a Sender learn of other clients.
- **Initialization:** How does a Recipient learn a client wants to communicate with them.
- **Operation:** How do participants exchange content.
and transport details are divided into 2 components:
- **Delivery Service:** How are payloads routed and delivered to a client.
- **Framing Strategy:** How are payloads encoded.
Defining these 5 parameters allows for chat protocol implementations to be fully defined, which allows clients from different applications to exchange messages.
While also dividing documents by their focus area.
## Abstract Transport
### Delivery service
A Delivery Service (DS) is the service or method that distributes payloads to clients. A DS accepts payloads with a delivery_address and delivers them to all subscribers of that delivery_address. Protocols use delivery_addresses to establish delivery contracts between senders and recipients. The mapping of delivery_addresses to DS-level concepts is implementation-specific.
#### Requirements
- A DS MUST provide a method for clients to subscribe to messages from a delivery_address
- Payloads sent to a delivery_address are delivered by a DS to all subscribers of that delivery_address
- A DS SHOULD handle segmentation if the underling transport limits message sizes
- A DS MAY NOT guarantee message delivery
- A DS MAY NOT guarantee message ordering
- A DS MAY reject payloads
### Framing Strategy
In this protocol framework, payloads from multiple protocols are potentially multiplexed over the same channel. This requires that clients are able to associate a given payload to a given instance of a protocol.
A framing strategy should define a common payload type as well as a method to determine which state machine a receiving client must use to decode it.
## Protocol Phases
In order to exchange content clients must be able to learn of each others existence, gather the pre-requisite information/parameters and, remain synchronized over time.
The lifecycle of a protocol instance is divided into three phases, which are described by a corresponding protocol.
- **Discovery Phase:** Discovery Protocol
- **Initialization Phase:** Initialization Protocol
- **Operation Phase:** Conversation Protocol
```mermaid
sequenceDiagram
participant D as ???
participant S as Saro
participant R as Raya
Note over D,S: Discovery
D -->> S: <IntroBundle>
Note over R,S: Initialization
S ->> R: Invite
Note over R,S: Operation
loop
par
R->> S: Send Message
and
S->> R: Send Message
end
end
```
### Discovery Protocol
A discovery protocol defines how clients gather the prerequisite information to contact another client.
The input requirements of the discovery protocol are not defined here, and largely determined by the desired user experience.
The output requirements of the discovery protocol are very implementation specific, and will depend on the initialization protocol requirements and DS chosen by the implementation. The data provided by the discovery protocol and required by the initialization protocol is called the `IntroductionBundle`.
- The discovery protocol MUST provide all data required for the initialization protocol.
Note: There is no requirement that the Discovery protocol be neither a complicated nor interactive process. Hypothetically If all required values to construct a IntroductionBundle could be statically defined, that would be sufficient for this definition.
### Initialization Protocol
A initialization protocol specifies how two clients can initiate communication. The input to this process is the `IntroductionBundle` and the output is an established instance of a `Conversation` between the participants.
The core of an initialization protocol is a defined location and procedure for receiving initial messages.
Many chat protocols choose to define the initialization protocol within the conversation protocol. This tight coupling produces two negative artifacts.
- New conversation protocols must define and deploy there own initialization channels. Increasing overhead and adding complexity.
- Protocol upgrades then create partitions in the communication network, as older clients have no means of communicating with new clients.
Separating channel initialization from conversation flow allows multiple conversations to reuse the same initialization channel. This reduces effort for new conversation protocols, and is especially valuable when upgrading existing ones. Being independent the initialization pathway can persist across conversation versions. Even if an older client cannot parse new message types, it can still recognize their presence, adding observability.
### Conversation Protocol
A conversation protocol defines how messages flow between participants, and subsequently determines the properties of that channel.
- A Conversation protocol MUST define the payloads it uses and how to handle them.
- A Conversation protocol SHOULD outline the cryptographic properties provided
- A Conversation protocol SHOULD describe bidirectional communication.
- A Conversation protocol ...
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
A list of references.

179
standards/application/contentframe.md Normal file
View File

@ -0,0 +1,179 @@
---
title: CONTENTFRAME
name:
category: Standards Track
tags:
editor: Jazzz
contributors:
---
## Abstract
This specification defines ContentFrame, a self-describing message format for decentralized chat networks.
ContentFrame wraps content payloads with metadata identifying their type and governing specification repository.
Using a `(domain, tag)` tuple, applications can uniquely identify message types and locate authoritative documentation for parsing unfamiliar content.
This approach enables permissionless innovation while maintaining the context needed for interoperability, allowing applications to gracefully handle messages from sources they don't explicitly know about.
## Motivation
In an interoperable chat network, participants cannot be assumed to use the same software to send and receive messages.
Users may employ different versions of the same application or different applications entirely.
This heterogeneity creates a fundamental challenge: how can applications support extensible message types without prior knowledge of every possible format?
Two naive approaches each have significant drawbacks:
**Developer-defined types** would allow flexibility but create fragmentation.
When developers define their own message types, the context for parsing these messages remains tightly coupled to the software that created them.
Other applications receiving these messages lack the necessary context to interpret them correctly.
This leads to multiple definitions of basic types such as `Text` and `Image` that are not compatible across applications.
**Fixed type systems** would ensure universal understanding but restrict innovation.
A predetermined set of message types eliminates ambiguity but adds friction for developers who want to extend functionality.
In a permissionless, decentralized protocol, requiring centralized approval for new message types contradicts core design principles.
The core challenge is managing fragmentation in a decentralized protocol while preserving developer freedom to innovate.
**Solution:** A self-describing message format that encodes both the payload and the metadata needed to parse it.
This approach directs application developers on how a message should be parsed while providing a clear path to learn about unfamiliar content types they encounter.
By decoupling the encoded data from the specific software that created it, applications can gracefully handle messages from diverse sources without sacrificing extensibility.
## Theory / Semantics
### ContentFrame
A ContentFrame provides a self-describing format for payload types by encoding both the type identifier and its administrative origin.
The core principle is that each payload should declare which entity is responsible for its definition and provide a unique type discriminator within that entity's namespace.
A ContentFrame consists of two key components:
- **Domain**: Points to a specification repository that defines and governs a collection of types
- **Tag**: A unique identifier within that domain that specifies which type the payload conforms to
Together, the tuple `(domain, tag)` serves two purposes:
1. **Identification**: Uniquely identifies the payload type without ambiguity
2. **Discovery**: Provides a path for developers to learn how to parse and support unfamiliar types
**Benefits:**
This approach provides several advantages for decentralized interoperability:
- **No naming collisions**: Developers can independently create types without coordinating with others, as each domain manages its own namespace
- **Type reuse**: Well-defined, established types can be shared across applications, reducing fragmentation
- **Graceful extensibility**: Applications encountering unknown types can direct developers to the authoritative specification
- **Decentralized governance**: No central authority is required to approve new types; domains manage their own specifications
By separating the "who defines this" (domain) from the "what is this" (tag), ContentFrame enables permissionless innovation while maintaining the context needed for interoperability.
### Concept Mapping
The following diagram illustrates the relationship between ContentFrame components and their specifications:
```mermaid
flowchart TD
d[Domain ID] -->|references| D
D[Domain] -->|Defines| T[Tag]
T -->|References| Specification
```
### Domain
A domain identifies the authority responsible for defining and governing a set of content types.
By including the domain, receiving applications can locate the authoritative specification for a type, regardless of which application originally sent it.
**Requirements:**
- A domain MUST be a valid URL as defined in [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986)
- A domain MUST host or reference definitions for all content types within its namespace
- A domain SHOULD be a specification repository or index that developers can reference
**Specification Format:**
Domains are responsible for describing their types in whatever format is most appropriate.
The only requirement is that the information needed to parse and understand each type is accessible from the domain URL.
**Domain ID Mapping:**
To minimize payload size, domains are mapped to integer identifiers.
Each domain is assigned a unique `domain_id` which is used in the wire format instead of the full URL.
- A `domain_id` MUST be a positive integer value
- A `domain_id` MUST correspond to exactly one unique domain
- The canonical mapping of `domain_id` to domains can be found in [Appendix A: Domains](#appendix-a-domains)
### Tag
A tag is a numeric identifier that uniquely specifies a content type within a domain's namespace.
After resolving the domain and tag, application developers have all the information needed to locate the definition and parse the payload.
**Requirements:**
- A tag MUST be a positive integer value
- A tag MUST uniquely identify a single type within its domain
- Two payloads with the same `(domain, tag)` tuple MUST conform to the same type specification
- A tag's meaning MUST NOT change after it has been assigned within a domain
**Domain Responsibility:**
Each domain is responsible for:
- Assigning and managing tag values within its namespace
- Documenting how each tag maps to a type specification
- Ensuring tag assignments remain stable and unambiguous
Tags are scoped to their domain, meaning the same tag value can represent different types in different domains without conflict.
## Wire Format Specification / Syntax
```protobuf
message ContentFrame {
uint32 domain_id = 1;
uint32 tag = 2;
bytes payload = 3;
}
```
**Field Descriptions:**
- **domain_id**: Identifies the domain that governs this content type
- **tag**: Identifies the specific content type within the domain's namespace
- **payload**: The encoded content data
All fields are required.
## Implementation Suggestions
### Tags to Specifications
Where possible, tag values should directly correspond to specification identifiers.
Using specification IDs as tags removes the need to maintain a separate mapping between tags and specifications.
### Fragmentation
This protocol allows multiple competing definitions of similar content types.
Having multiple definitions of `Text` or `Image` increases fragmentation between applications.
Where possible, reusing existing types will reduce burden on developers and increase interoperability.
Domains should focus on providing types unique to their service or use case.
# Appendix A: Domains
![TODO] Find appropriate home for this registry.
Domain IDs are assigned sequentially on a first-come, first-served basis.
New domains are added via pull request.
**Registry Rules:**
- A domain MUST only appear once in the table
- A domain MAY be updated by the original submitter if the repository has been moved
**Registry:**
| domain_id | specification repository |
|-----------|--------------------------------------|
| 1 | https://github.com/waku-org/specs |

153
standards/application/introduction_encoding.md Normal file
View File

@ -0,0 +1,153 @@
---
title: Introduction Bundle Encoding
name: introduction-bundle-encoding
category: Standards Track
tags: core, encoding
editor:
contributors:
- Patryk <patryk@status.im>
- Jazzz <jazz@status.im>
---
## Abstract
This specification defines the encoding format for Introduction Bundles — the
out-of-band information shared so that a remote party can initiate contact with
the bundle publisher.
## Background / Rationale / Motivation
Users need a way to share contact information across arbitrary channels
(messaging apps, emails, QR codes, URLs, CLI terminals) without relying on a
centralized directory. The Introduction Bundle provides the cryptographic
material required to establish an encrypted conversation.
The encoding must be:
- Copy-paste safe across arbitrary text transports.
- Space-efficient for manual sharing.
- Version-aware to support protocol evolution.
## Theory / Semantics
An Introduction Bundle is bound to a specific protocol version; its encoded
form identifies the version unambiguously.
### Why ASCII, Not Unicode
The encoded string consists entirely of printable ASCII. This is a natural
consequence of the design rather than a defensive choice against Unicode:
1. **No human-readable content.** The format contains a fixed prefix, a numeric
version, and a binary-to-text encoded payload. These components do not
require characters outside ASCII. Common binary-to-text encodings such as
hex, base32, and base64 produce ASCII output.
2. **Encoding stability.** Printable ASCII characters are represented
identically in widely deployed text encodings such as UTF-8 and Latin-1,
avoiding ambiguity in character interpretation across transports.
3. **Restricted visible alphabet.** Limiting the character set reduces the risk
of visually confusable or non-rendering characters during manual comparison,
transcription, or copy-paste across different platforms, terminals, and
fonts.
### Delimiter Choice
The `_` character is present in the alphabets of common binary-to-text
encodings (hex, base32, base64url) and may therefore appear inside the payload.
Parsers MUST split the encoded string on the **first three** `_` characters;
everything after the third `_` is the payload verbatim.
The underscore was chosen because it is a printable ASCII character that is
safe in URLs, filenames, and plain text without requiring escaping. Delimiters
outside this category (e.g., `.` or `:`) may require percent-encoding in
certain URL contexts, undermining the transport-safety goal.
## Wire Format Specification / Syntax
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC 2119].
### Text Format
The Introduction Bundle is encoded as a single ASCII string:
```
logos_<namespace>_<version>_<payload>
```
| Field | Description |
|-------------|-------------------------------------------------------------------|
| `logos` | Fixed prefix. Identifies the string as a Logos protocol artifact. |
| `namespace` | Domain identifier. Distinguishes bundle types. |
| `version` | Protocol version number. Decimal integer, no leading zeros. |
| `payload` | Binary-to-text encoded payload (version-specific). |
Fields are separated by `_` (underscore).
### V1
#### Parameters
| Parameter | Value |
|-------------|--------------------------------|
| `namespace` | `chatintro` |
| `version` | `1` |
| `encoding` | base64url, no padding |
| `payload` | Protobuf-encoded `IntroBundle` |
#### Payload Encoding
V1 uses **base64url without padding** ([RFC 4648 §5], padding
characters `=` omitted). This encoding was chosen for:
- URL safety without percent-encoding.
- ~33% size overhead (compared to ~100% for hex).
- Wide library support across languages.
Future versions MAY choose a different binary-to-text encoding.
#### Binary Payload
The payload is a Protocol Buffers (proto3) encoding of:
```proto
syntax = "proto3";
message IntroBundle {
bytes installation_pubkey = 1; // 32 bytes, X25519
bytes ephemeral_pubkey = 2; // 32 bytes, X25519
bytes signature = 3; // 64 bytes, XEdDSA
}
```
The encoding MUST use standard proto3 serialization. Canonical (deterministic)
serialization is NOT REQUIRED; decoders MUST accept any valid proto3 encoding
of the message.
#### Encoding Procedure
1. Construct the `IntroBundle` protobuf message.
2. Serialize using proto3 encoding (~134 bytes).
3. Encode as base64url without padding (~179 characters).
4. Prepend the preamble with version prefix.
The resulting string is ~197 printable ASCII characters.
## Security/Privacy Considerations
The signature prevents tampering but does not provide confidentiality. Bundles
should be transmitted over channels appropriate for the user's threat model.
## Copyright
Copyright and related rights waived via [CC0].
## References
- [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt) — Key words for use in RFCs
- [RFC 4648 §5](https://www.rfc-editor.org/rfc/rfc4648#section-5) — Base 64 Encoding with URL and Filename Safe Alphabet
- [CC0](https://creativecommons.org/publicdomain/zero/1.0/) — Creative Commons Zero Public Domain Dedication
[RFC 2119]: https://www.ietf.org/rfc/rfc2119.txt
[RFC 4648 §5]: https://www.rfc-editor.org/rfc/rfc4648#section-5
[CC0]: https://creativecommons.org/publicdomain/zero/1.0/

605
standards/application/messaging-api.md Normal file
View File

@ -0,0 +1,605 @@
---
title: MESSAGING-API
name: Messaging API definition
category: Standards Track
status: raw
tags: [reliability, application, api, protocol composition]
editor: Oleksandr Kozlov <oleksandr@status.im>
contributors:
- Oleksandr Kozlov <oleksandr@status.im>
- Prem Chaitanya Prathi <prem@status.im>
- Franck Royer <franck@status.im>
---
## Table of contents
<!-- TOC -->
* [Table of contents](#table-of-contents)
* [Abstract](#abstract)
* [Motivation](#motivation)
* [Syntax](#syntax)
* [API design](#api-design)
* [IDL](#idl)
* [Primitive types and general guidelines](#primitive-types-and-general-guidelines)
* [Language mappings](#language-mappings)
* [Application](#application)
* [The Messaging API](#the-messaging-api)
* [Common](#common)
* [Common type definitions](#common-type-definitions)
* [Init node](#init-node)
* [Init node type definitions](#init-node-type-definitions)
* [Init node function definitions](#init-node-function-definitions)
* [Init node predefined values](#init-node-predefined-values)
* [Init node extended definitions](#init-node-extended-definitions)
* [Messaging](#messaging)
* [Messaging type definitions](#messaging-type-definitions)
* [Messaging function definitions](#messaging-function-definitions)
* [Messaging extended definitions](#messaging-extended-definitions)
* [Subscriptions](#subscriptions)
* [Subscriptions type definitions](#subscriptions-type-definitions)
* [Subscriptions function definitions](#subscriptions-function-definitions)
* [Subscriptions extended definitions](#subscriptions-extended-definitions)
* [Health](#health)
* [Health type definitions](#health-type-definitions)
* [Health function definitions](#health-function-definitions)
* [Health extended definitions](#health-extended-definitions)
* [The Validation API](#the-validation-api)
* [Security/Privacy Considerations](#securityprivacy-considerations)
* [Copyright](#copyright)
<!-- TOC -->
## Abstract
This document specifies an Application Programming Interface (API) that is RECOMMENDED for developers of the [WAKU2](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/10/waku2.md) clients to implement,
and for consumers to use as a single entry point to its functionalities.
This API defines the RECOMMENDED interface for leveraging Logos Messaging protocols to send and receive messages.
Application developers SHOULD use it to access capabilities for peer discovery, message routing, and peer-to-peer reliability.
## Motivation
The accessibility of Logos Messaging protocols is capped by the accessibility of their implementations, and hence API.
This RFC enables a concerted effort to draft an API that is simple and accessible, and provides an opinion on sane defaults.
The API defined in this document is an opinionated-by-purpose method to use the more agnostic [WAKU2](https://lip.logos.co/messaging/standards/core/10/waku2.html) protocols.
## Syntax
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
## API design
### IDL
A custom Interface Definition Language (IDL) in YAML is used to define the Messaging API.
Existing IDL Such as OpenAPI, AsyncAPI or WIT do not exactly fit the requirements for this API.
Hence, instead of having the reader learn a new IDL, we propose to use a simple IDL with self-describing syntax.
An alternative would be to choose a programming language. However, such choice may express unintended opinions on the API.
### Primitive types and general guidelines
- No `default` means that the value is mandatory, meaning a `default` value implies an optional parameter.
- Primitive types are `string`, `int`, `bool`, `byte`, `enum` and `uint`
- Complex pre-defined types are:
- `object`: object and other nested types.
- `array`: iterable object containing values of all the same type. Syntax: `array<T>` where `T` is the element type (e.g., `array<string>`, `array<byte>`).
- `result`: an enum type that either contains a value or void (success), or an error (failure); The error is left to the implementor.
- `error`: Left to the implementor on whether `error` types are `string` or `object` in the given language.
- `event_emitter`: an object that emits events with specific event names and associated event data types.
- Usage of `result` is RECOMMENDED, usage of exceptions is NOT RECOMMENDED, no matter the language.
TODO: Review whether to specify categories of errors.
### Language mappings
How the API definition should be translated to specific languages.
```yaml
language_mappings:
typescript:
naming_convention:
- functions: "camelCase"
- variables: "camelCase"
- types: "PascalCase"
event_emitter: "Use EventEmitter object with `emit`, `addListener`, etc; with event name the string specified in IDL. For example. eventEmitter.emit('message:sent',...)"
nim:
naming_convention:
- functions: "camelCase"
- variables: "camelCase"
- types: "PascalCase"
event_emitter: TBD
```
### Application
This API is designed for generic use and ease across all programming languages, for `edge` and `core` type nodes.
## The Messaging API
```yaml
api_version: "0.0.1"
library_name: "liblogosdelivery"
description: "Logos Messaging: a private and censorship-resistant message routing library."
```
### Common
This section describes common types used throughout the API.
Note that all types in the API are described once in this document, in a single section. Types should just forward-reference other types when needed.
#### Common type definitions
```yaml
types:
WakuNode:
type: object
description: "A node instance."
fields:
messageEvents:
type: MessageEvents
description: "The node's messaging event emitter"
healthEvents:
type: HealthEvents
description: "The node's health monitoring event emitter"
RequestId:
type: string
description: "A unique identifier for a request"
```
### Init node
#### Init node type definitions
```yaml
types:
NodeConfig:
type: object
fields:
mode:
type: string
constraints: [ "edge", "core" ]
default: "core" # "edge" for mobile and browser devices.
description: "The mode of operation of the node; 'edge' of the network: relies on other nodes for message routing; 'core' of the network: fully participate to message routing."
protocols_config:
type: ProtocolsConfig
default: TheWakuNetworkPreset
networking_config:
type: NetworkConfig
default: DefaultNetworkingConfig
eth_rpc_endpoints:
type: array<string>
description: "Eth/Web3 RPC endpoint URLs, only required when RLN is used for message validation; fail-over available by passing multiple URLs. Accepting an object for ETH RPC will be added at a later stage."
ProtocolsConfig:
type: object
fields:
entry_nodes:
type: array<string>
default: []
description: "Nodes to connect to; used for discovery bootstrapping and quick connectivity. enrtree and multiaddr formats are accepted. If not provided, node does not bootstrap to the network (local dev)."
static_store_nodes:
type: array<string>
default: []
# TODO: confirm behaviour at implementation time.
description: "The passed nodes are prioritised for store queries."
cluster_id:
type: uint
description: "The cluster ID for the network. Cluster IDs are defined in [RELAY-SHARDING](https://github.com/logos-messaging/specs/blob/master/standards/core/relay-sharding.md) and allocated in [RELAY-STATIC-SHARD-ALLOC](https://github.com/logos-messaging/specs/blob/master/informational/relay-static-shard-alloc.md)."
auto_sharding_config:
type: AutoShardingConfig
default: DefaultAutoShardingConfig
description: "The auto-sharding config, if sharding mode is `auto`"
message_validation:
type: MessageValidation
description: "If the default config for TWN is not used, then we still provide default configuration for message validation."
default: DefaultMessageValidation
NetworkingConfig:
type: object
fields:
listen_ipv4:
type: string
default: "0.0.0.0"
description: "The network IP address on which libp2p and discv5 listen for inbound connections. Not applicable for some environments such as the browser."
p2p_tcp_port:
type: uint
default: 60000
description: "The TCP port used for libp2p, relay, etc aka, general p2p message routing. Not applicable for some environments such as the browser."
discv5_udp_port:
type: uint
default: 9000
description: "The UDP port used for discv5. Not applicable for some environments such as the browser."
AutoShardingConfig:
type: object
fields:
num_shards_in_cluster:
type: uint
description: "The number of shards in the configured cluster; this is a globally agreed value for each cluster."
MessageValidation:
type: object
fields:
max_message_size:
type: string
default: "150 KiB"
description: "Maximum message size. Accepted units: KiB, KB, and B. e.g. 1024KiB; 1500 B; etc."
# For now, RLN is the only message validation available
rln_config:
type: RlnConfig
# If the default config for TWN is not used, then we do not apply RLN
default: none
RlnConfig:
type: object
fields:
contract_address:
type: string
description: "The address of the RLN contract that exposes `root` and `getMerkleRoot` ABIs"
chain_id:
type: uint
description: "The chain ID on which the RLN contract is deployed"
epoch_size_sec:
type: uint
description: "The epoch size to use for RLN, in seconds"
```
#### Init node function definitions
```yaml
functions:
createNode:
description: "Initialises a node instance"
parameters:
- name: nodeConfig
type: NodeConfig
description: "The node configuration."
returns:
type: result<WakuNode, error>
```
#### Init node predefined values
```yaml
values:
DefaultNetworkingConfig:
type: NetworkConfig
fields:
listen_ipv4: "0.0.0.0"
p2p_tcp_port: 60000
discv5_udp_port: 9000
TheWakuNetworkPreset:
type: ProtocolsConfig
fields:
entry_nodes: [ "enrtree://AIRVQ5DDA4FFWLRBCHJWUWOO6X6S4ZTZ5B667LQ6AJU6PEYDLRD5O@sandbox.waku.nodes.status.im" ]
# On TWN, we encourage the usage of discovered store nodes
static_store_nodes: []
cluster_id: 1
auto_sharding_config:
type: AutoShardingConfig
fields:
num_shards_in_cluster: 8
message_validation: TheWakuNetworkMessageValidation
TheWakuNetworkMessageValidation:
type: MessageValidation
fields:
max_message_size: "150 KiB"
rln_config:
type: RlnConfig
fields:
contract_address: "0xB9cd878C90E49F797B4431fBF4fb333108CB90e6"
chain_id: 59141
epoch_size_sec: 600 # 10 minutes
# If not preset is used, autosharding on one cluster is applied by default
# This is a safe default that abstract shards (content topic shard derivation), and it enables scaling at a later stage
DefaultAutoShardingConfig:
type: AutoShardingConfig
fields:
num_shards_in_cluster: 1
# If no preset is used, we only apply a max size limit to messages
DefaultMessageValidation:
type: MessageValidation
fields:
max_message_size: "150 KiB"
rln_config: none
```
#### Init node extended definitions
**`mode`**:
If the `mode` set is `edge`, the initialised `WakuNode` SHOULD use:
- [LIGHTPUSH](https://lip.logos.co/messaging/standards/core/19/lightpush.html) as client
- [FILTER](https://lip.logos.co/messaging/standards/core/12/filter.html) as client
- [STORE](https://lip.logos.co/messaging/standards/core/13/store.html) as client
- [METADATA](https://lip.logos.co/messaging/standards/core/66/metadata.html) as client
- [PEER-EXCHANGE](https://lip.logos.co/messaging/standards/core/34/peer-exchange.html) as client
- [P2P-RELIABILITY](/standards/application/p2p-reliability.md)
If the `mode` set is `core`, the initialised `WakuNode` SHOULD use:
- [RELAY](https://lip.logos.co/messaging/standards/core/11/relay.html)
- [LIGHTPUSH](https://lip.logos.co/messaging/standards/core/19/lightpush.html) as service node
- [FILTER](https://lip.logos.co/messaging/standards/core/12/filter.html) as service node
- [STORE](https://lip.logos.co/messaging/standards/core/13/store.html) as client
- [METADATA](https://lip.logos.co/messaging/standards/core/66/metadata.html) as client and service node
- [P2P-RELIABILITY](/standards/application/p2p-reliability.md)
- [DISCV5](https://lip.logos.co/messaging/standards/core/33/discv5.html)
- [PEER-EXCHANGE](https://lip.logos.co/messaging/standards/core/34/peer-exchange.html) as client and service node
- [RENDEZVOUS](https://github.com/logos-messaging/specs/blob/master/standards/core/rendezvous.md) as client and service node
`edge` mode SHOULD be used if node functions in resource restricted environment,
whereas `core` SHOULD be used if node has no strong hardware or bandwidth restrictions.
### Messaging
#### Messaging type definitions
```yaml
types:
MessageEnvelope:
type: object
fields:
content_topic:
type: string
description: "Content-based filtering field as defined in [TOPICS](https://lip.logos.co/messaging/informational/23/topics.html#content-topics)"
payload:
type: array<byte>
description: "The message data."
ephemeral:
type: bool
default: false
description: "Whether the message is ephemeral. Read at [ATTRIBUTES](https://lip.logos.co/messaging/standards/core/14/message.html#message-attributes)"
MessageReceivedEvent:
type: object
description: "Event emitted when a message is received from the network"
fields:
message:
type: MessageEnvelope
description: "The received message's payload and metadata"
MessageSentEvent:
type: object
description: "Event emitted when a message is sent to the network"
fields:
request_id:
type: RequestId
description: "The request ID associated with the sent message"
message_hash:
type: string
description: "Hash of the message that got sent to the network"
MessageSendErrorEvent:
type: object
description: "Event emitted when a message send operation fails"
fields:
request_id:
type: RequestId
description: "The request ID associated with the failed message"
message_hash:
type: string
description: "Optional property. Hash of the message that got error"
error:
type: string
description: "Error message describing what went wrong"
MessageSendPropagatedEvent:
type: object
description: "Confirmation that a message has been correctly delivered to some neighbouring nodes."
fields:
request_id:
type: RequestId
description: "The request ID associated with the propagated message in the network"
message_hash:
type: string
description: "Hash of the message that got propagated within the network"
MessageEvents:
type: event_emitter
description: "Event source for message-related events"
events:
"message:received":
type: MessageReceivedEvent
"message:sent":
type: MessageSentEvent
"message:send-error":
type: MessageSendErrorEvent
"message:send-propagated":
type: MessageSendPropagatedEvent
```
#### Messaging function definitions
```yaml
functions:
send:
description: "Send a message through the network."
parameters:
- name: message
type: MessageEnvelope
description: "Parameters for sending the message."
returns:
type: result<RequestId, error>
```
#### Messaging extended definitions
A first `message` sent with a certain `contentTopic` SHOULD trigger a subscription for such `contentTopic` as described in the `Subscriptions` section.
The node uses [P2P-RELIABILITY](/standards/application/p2p-reliability.md) strategies to ensure message delivery.
### Subscriptions
#### Subscriptions type definitions
```yaml
types:
SubscriptionError:
type: object
description: "A content topic subscription-related operation failed synchronously and irremediably"
fields:
content-topic:
type: string
description: "Content topic that the node failed to subscribe to or unsubscribe from"
error:
type: string
description: "Error message describing what went wrong"
```
#### Subscriptions function definitions
```yaml
functions:
subscribe:
description: "Subscribe to specific content topics"
parameters:
- name: contentTopics
type: Array<string>
description: "The content topics for the node to subscribe to."
returns:
type: result<void, array<SubscriptionError>>
unsubscribe:
description: "Unsubscribe from specific content topics"
parameters:
- name: contentTopics
type: Array<ContentTopic>
description: "The content topics for the node to unsubscribe from."
returns:
type: result<void, array<SubscriptionError>>
```
#### Subscriptions extended definitions
**`mode`**:
If the `mode` set is `edge`, `subscribe` SHOULD trigger set up a subscription using [FILTER](https://lip.logos.co/messaging/standards/core/12/filter.html) as client and [P2P-RELIABILITY](/standards/application/p2p-reliability.md).
If the `mode` set is `core`, `subscribe` SHOULD trigger set up a subscription using [RELAY](https://lip.logos.co/messaging/standards/core/11/relay.html) and [P2P-RELIABILITY](/standards/application/p2p-reliability.md).
This MAY trigger joining a new shard if not already set.
Only messages on subscribed content topics SHOULD be emitted by a `MessageEvents` event source, meaning messages received via `RELAY` SHOULD be filtered by content topics before emission.
**`error`**:
Only irremediable failures should lead to synchronously returning a subscription error for failed subscribe or unsubscribe operations.
Failure to reach nodes can be omitted, and should be handled via the health events;
[P2P-RELIABILITY](/standards/application/p2p-reliability.md) SHOULD handle automated re-subscriptions and redundancy.
Examples of irremediable failures are:
- Invalid content topic format
- Exceeding number of content topics
- Node not started
- Already unsubscribed
- Other node-level configuration issue
### Health
#### Health type definitions
```yml
types:
ConnectionStatus:
type: enum
values: [Disconnected, PartiallyConnected, Connected]
description: "Used to identify health of the operating node"
HealthConnectionStatusEvent:
type: object
description: "Event emitted when the overall node health status changes"
fields:
connection-status:
type: ConnectionStatus
description: "The node's new connection status"
HealthEvents:
type: event_emitter
description: "Event source for health-related events."
events:
"health:connection-status":
type: HealthConnectionStatusEvent
```
#### Health function definitions
TODO
#### Health extended definitions
`Disconnected` indicates that the node has lost connectivity for message reception,
sending, or both, and as a result, it cannot reliably receive or transmit messages.
`PartiallyConnected` indicates that the node meets the minimum operational requirements:
it is connected to at least one peer with a protocol to send messages ([LIGHTPUSH](https://lip.logos.co/messaging/standards/core/19/lightpush.html) or [RELAY](https://lip.logos.co/messaging/standards/core/11/relay.html)),
one peer with a protocol to receive messages ([FILTER](https://lip.logos.co/messaging/standards/core/12/filter.html) or [RELAY](https://lip.logos.co/messaging/standards/core/11/relay.html)),
and one peer with [STORE](https://lip.logos.co/messaging/standards/core/13/store.html) service capabilities,
although performance or reliability may still be impacted.
`Connected` indicates that the node is operating optimally,
with full support for message reception and transmission.
### Debug
#### Debug function definitions
```yaml
functions:
getAvailableNodeInfoIds:
description: "Returns a list of available node information identifiers. e.g., [ version, my_peer_id, metrics ]."
returns:
type: result<array<string>, error>
getNodeInfo:
description: "Returns the JSON formatted node's information that is requested. Expect single value or list results depending on requested information."
parameters:
- name: nodeInfoId
type: string
description: "Information identifier. The only supported values are the ones returned by getAvailableNodeInfoItems function."
returns:
type: result<string, error>
getAvailableConfigs:
description: "Returns a list of all available options, their description and default values."
returns:
type: string
```
## The Validation API
[WAKU2-RLN-RELAY](https://lip.logos.co/messaging/standards/core/17/rln-relay.html) is currently the primary message validation mechanism in place.
Work is scheduled to specify a validate API to enable plug-in validation.
As part of this API, it will be expected that a validation object can be passed,
that would contain all validation parameters including RLN.
In the time being, parameters specific to RLN are accepted for the message validation.
RLN can also be disabled.
## Security/Privacy Considerations
See [WAKU2-ADVERSARIAL-MODELS](https://github.com/logos-messaging/specs/blob/master/informational/adversarial-models.md).
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

449
standards/application/reliable-channel-api.md Normal file
View File

@ -0,0 +1,449 @@
---
title: RELIABLE-CHANNEL-API
name: Reliable Channel API definition
category: Standards Track
status: raw
tags: [reliability, application, api, sds, segmentation]
editors:
- Igor Sirotin <sirotin@status.im>
- Jazz Turner-Baggs <jazz@status.im>
- Ivan Folgueira Bande <ivansete@status.im>
---
## Table of contents
<!-- TOC -->
* [Table of contents](#table-of-contents)
* [Abstract](#abstract)
* [Motivation](#motivation)
* [Syntax](#syntax)
* [API design](#api-design)
* [Architectural position](#architectural-position)
* [IDL](#idl)
* [Components](#components)
* [Segmentation](#segmentation)
* [Scalable Data Sync (SDS)](#scalable-data-sync-sds)
* [Rate Limit Manager](#rate-limit-manager)
* [Encryption Hook](#encryption-hook)
* [Procedures](#procedures)
* [Node initialization](#node-initialization)
* [Outgoing message processing](#outgoing-message-processing)
* [Incoming message processing](#incoming-message-processing)
* [Rate limiting](#rate-limiting)
* [Encryption](#encryption)
* [The Reliable Channel API](#the-reliable-channel-api)
* [Channel lifecycle](#channel-lifecycle)
* [Channel usage](#channel-usage)
* [Node configuration](#node-configuration)
* [Type definitions](#type-definitions)
* [Security/Privacy Considerations](#securityprivacy-considerations)
* [Copyright](#copyright)
<!-- TOC -->
## Abstract
This document specifies the **Reliable Channel API**,
an application-level interface that sits between the application layer and the [MESSAGING-API](/standards/application/messaging-api.md) plus [P2P-RELIABILITY](/standards/application/p2p-reliability.md), i.e., `application` <-> **reliable-channel-api** <-> `messaging-api/p2p-reliability`.
It bundles segmentation, end-to-end reliability via [Scalable Data Sync (SDS)](https://lip.logos.co/ift-ts/raw/sds.html), rate limit management, and a pluggable encryption hook
into a single interface for sending and receiving messages reliably.
## Motivation
The [MESSAGING-API](/standards/application/messaging-api.md) provides peer-to-peer reliability via [P2P-RELIABILITY](/standards/application/p2p-reliability.md),
but does not provide high end-to-end delivery guarantees from sender to recipient.
This API addresses that gap by introducing:
- **[SEGMENTATION](/standards/application/segmentation.md)** to handle large messages exceeding network size limits.
- **[SDS](https://lip.logos.co/ift-ts/raw/sds.html)** to provide causal-history-based end-to-end acknowledgement and retransmission.
- **Rate Limit Manager** to comply with [RLN](https://lip.logos.co/messaging/standards/core/17/rln-relay.html) constraints when sending segmented messages.
- **Encryption Hook** to allow upper layers to provide a pluggable encryption mechanism. This enables applications to provide Confidentiality and Integrity if desired.
The separation between Reliable Channels and encryption ensures the API remains agnostic to identity and key management concerns,
which are handled by higher layers.
## Syntax
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT",
"RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
## API design
### Architectural position
The Reliable Channel API sits between the application layer and the Messaging API, as follows:
```
┌────────────────────────────────────────────────────────────┐
│ Application Layer │
└───────────────────────────┬────────────────────────────────┘
│
┌───────────────────────────▼────────────────────────────────┐
│ Reliable Channel API │
│ ┌──────────────┐ ┌─────┐ ┌───────────────┐ ┌──────────┐ │
│ │ Segmentation │ │ SDS │ │ Rate Limit Mgr│ │Encryption│ │
│ │ │ │ │ │ │ │ Hook │ │
│ └──────────────┘ └─────┘ └───────────────┘ └──────────┘ │
└───────────────────────────┬────────────────────────────────┘
│
┌───────────────────────────▼────────────────────────────────┐
│ Messaging API │
│ (P2P Reliability, Relay, Filter, Lightpush, Store) │
└────────────────────────────────────────────────────────────┘
```
### IDL
A custom Interface Definition Language (IDL) in YAML is used, consistent with [MESSAGING-API](/standards/application/messaging-api.md).
## Components
### Segmentation
A protocol that splits message payloads into smaller units during transmission and reassembles them upon reception. The component is instantiated by supplying the appropriate value to SegmentationConfig.
See [SEGMENTATION](./segmentation.md).
### Scalable Data Sync (SDS)
[SDS](https://lip.logos.co/ift-ts/raw/sds.html) provides end-to-end delivery guarantees using causal history tracking.
- Each new segment to be sent, requires the following data:
- `MessageId`: a keccak-256([message](https://lip.logos.co/ift-ts/raw/sds.html#message)'s content) hex string (e.g. `4e03657aea45a94fc7d47ba826c8d667c0d1e6e33a64a036ec44f58fa12d6c45`), generated by the Reliable Channel API.
- `ChannelId`: the `channelId` passed to `createReliableChannel`.
- `Retrieval hint`: the transport `MessageHash` of previous segments, mentioned as `message_hash` in [MESSAGING-API](/standards/application/messaging-api.md), upon ``MessageSendPropagatedEvent` reception.
The hint provider registered during [Node initialization](#node-initialization) performs this `MessageId → MessageHash` lookup. In turn, that mapping MUST be persisted by SDS using the `persistence` backend configured in `SdsConfig`.
- Each sent segment is added to an outgoing buffer.
- The recipient sends acknowledgements back to the sender upon receiving segments.
- The sender removes acknowledged segments from the outgoing buffer.
- Unacknowledged segments are retransmitted after `acknowledgementTimeoutMs`.
- SDS state MUST be persisted using the `persistence` backend configured in `SdsConfig`.
### Rate Limit Manager
The Rate Limit Manager ensures compliance with [RLN](https://lip.logos.co/messaging/standards/core/17/rln-relay.html) rate constraints.
- It tracks how many messages have been sent in the current epoch.
- When the limit is approached, segment dispatch MUST be delayed to the next epoch.
- The epoch size MUST match the `[epochPeriodSec](https://lip.logos.co/messaging/standards/core/17/rln-relay.html#epoch-length)` configured in `RateLimitConfig`.
### Encryption Hook
The Encryption Hook provides a pluggable interface for upper layers to inject encryption.
- The hook is optional; when not provided, messages are sent unencrypted.
- Encryption is applied per segment, after segmentation and SDS.
- Decryption is applied per segment before being processed by SDS.
- The `Encryption` interface MUST be implemented by the caller when the hook is provided.
- The Reliable Channel API MUST NOT impose any specific encryption scheme.
## Procedures
### Node initialization
When a node is created via `createNode` (defined in [MESSAGING-API](/standards/application/messaging-api.md)),
the implementation MUST perform the following setup before the node is used:
1. **Configure SDS persistence**: Supply the `Persistence` backend from `SdsConfig` to the SDS module so that
causal history and outgoing buffers survive restarts.
2. **Configure SDS hint provider**: Register a hint provider with the SDS module.
The hint provider converts an SDS `MessageId` into its corresponding `MessageHash`.
3. **Configure Segmentation persistence**: Supply the `Persistence` backend from `SegmentationConfig` to the
[Segmentation](./segmentation.md) module so that partially reassembled messages survive restarts.
4. **Fetch missed messages**: Retrieve messages missed while offline as described in
[MESSAGING-API — Fetching missed messages on startup](/standards/application/messaging-api.md#init-node-extended-definitions).
### Outgoing message processing
When `send` is called, the implementation MUST process `message` in the following order:
1. **Segment**: Split the payload into segments as defined in [SEGMENTATION](./segmentation.md).
2. **Apply [SDS](https://lip.logos.co/ift-ts/raw/sds.html)**: add each sds message to the SDS outgoing buffer (see [SDS](#scalable-data-sync-sds) for parameter bindings).
3. **Encrypt**: If an `Encryption` implementation is provided, encrypt each segment before transmission.
4. **Rate Limit**: If `RateLimitConfig.enabled` is `true`, delay dispatch as needed to comply with [RLN](https://lip.logos.co/messaging/standards/core/17/rln-relay.html) epoch constraints.
5. **Dispatch**: Send each segment via the underlying [MESSAGING-API](/standards/application/messaging-api.md).
### Incoming message processing
When a segment is received from the network, the implementation MUST process it in the following order:
1. **Decrypt**: If an `Encryption` implementation is provided, decrypt the segment.
2. **Apply [SDS](https://lip.logos.co/ift-ts/raw/sds.html)**: Deliver the segment to the SDS layer, which emits acknowledgements and detects gaps.
- **Detect missing dependencies**: If SDS detects a gap in the causal history, it MUST make a best-effort attempt to retrieve the missing message. The `Retrieval hint` (see [Scalable Data Sync (SDS)](#scalable-data-sync-sds)) carried in each SDS message provides the transport `MessageHash` needed to query the store; without it, store retrieval is not possible. If the message cannot be retrieved, SDS MAY mark it as lost.
3. **Reassemble**: Once all segments for a message have been received, reassemble and emit a `reliable:message:received` event.
### Rate limiting
When `RateLimitConfig.enabled` is `true`, the implementation MUST space segment transmissions
to comply with the [RLN](https://lip.logos.co/messaging/standards/core/17/rln-relay.html) epoch constraints defined in `[epochPeriodSec](https://lip.logos.co/messaging/standards/core/17/rln-relay.html#epoch-length)`.
Segments MUST NOT be sent at a rate that would violate the RLN message rate limit for the active epoch.
### Encryption
The `encryption` parameter in `createReliableChannel` is intentionally optional.
The Reliable Channel API is agnostic to encryption mechanisms.
When an `Encryption` implementation is provided, it MUST be applied as described in [Outgoing message processing](#outgoing-message-processing) and [Incoming message processing](#incoming-message-processing).
## The Reliable Channel API
This API considers the types defined by [MESSAGING-API](/standards/application/messaging-api.md) plus the following.
### Channel lifecycle
This point assumes that a WakuNode instance is created beforehand. See `createNode` function
in [MESSAGING-API](/standards/application/messaging-api.md).
```yaml
functions:
createReliableChannel:
description: "Creates a reliable channel over the given content topic. Sets up the required SDS state,
segmentation, and encryption, and subscribes to `contentTopic`."
parameters:
- name: node
type: WakuNode
description: "The underlying messaging node, as defined in [MESSAGING-API](/standards/application/messaging-api.md).
Used to send segments and to subscribe/unsubscribe to the content topics."
- name: channelId
type: string
description: "Unique identifier for this channel. Represents the reliable (SDS), segmented, and optionally-encrypted session."
- name: contentTopic
type: string
description: "The topic this channel listens and sends on. This has routing and filtering connotations."
- name: senderId
type: string
description: "An identifier for this sender. SHOULD be unique and persisted between sessions."
- name: encryption
type: optional<Encryption>
default: none
description: "Optional pluggable encryption implementation. If none, messages are sent unencrypted."
returns:
type: result<ReliableChannel, error>
closeChannel:
description: "Closes a reliable channel, releases all associated resources and internal state,
and unsubscribes from its content topic via the underlying [MESSAGING-API](/standards/application/messaging-api.md)."
parameters:
- name: channel
type: ReliableChannel
description: "The channel handle returned by `createReliableChannel`."
returns:
type: result<void, error>
```
### Channel usage
```yaml
functions:
send:
description: "Send a message through a reliable channel. The message is always segmented,
SDS-tracked, rate-limited (optional), and encrypted (optional)."
parameters:
- name: channel
type: ReliableChannel
description: "The channel handle returned by `createReliableChannel`."
- name: message
type: array<byte>
description: "The raw message payload to send."
returns:
type: result<RequestId, error>
description: "Returns a `RequestId` that callers can use to correlate subsequent `MessageSentEvent` or `MessageSendErrorEvent` events."
```
### Node configuration
This spec extends `NodeConfig`, needed to create a node, which is
defined in [MESSAGING-API](/standards/application/messaging-api.md),
with `sds_config` and `rate_limit_config` fields.
```yaml
NodeConfig: # Extends NodeConfig defined in MESSAGING-API
fields:
sds_config:
type: SdsConfig
description: "SDS configuration. See SdsConfig defined in this spec."
```
### Type definitions
```yaml
types:
ReliableChannel:
type: object
description: "A handle representing an open reliable channel.
Returned by `createReliableChannel` and used to send messages and receive events.
Internal state (SDS, segmentation, encryption) is managed by the implementation."
events:
"reliable:message:received":
type: MessageReceivedEvent
"reliable:message:sent":
type: MessageSentEvent
"reliable:message:delivered":
type: MessageDeliveredEvent
"reliable:message:send-error":
type: MessageSendErrorEvent
"reliable:message:delivery-error":
type: MessageDeliveryErrorEvent
MessageReceivedEvent:
type: object
description: "Event emitted when a complete message has been received and reassembled."
fields:
message:
type: array<byte>
description: "The reassembled message payload."
MessageSentEvent:
type: object
description: "Event emitted when all segments of a message have been transmitted to the network.
This confirms network-level dispatch only; it does not guarantee the recipient has processed the message.
For end-to-end confirmation, listen for `MessageDeliveredEvent`."
fields:
requestId:
type: RequestId
description: "The identifier of the `send` operation whose segments have all been dispatched to the network."
MessageDeliveredEvent:
type: object
description: "Event emitted when the recipient has confirmed end-to-end receipt of a message via SDS acknowledgements.
This event is fired asynchronously after `MessageSentEvent`, once the SDS layer receives explicit acknowledgements from the recipient."
fields:
requestId:
type: RequestId
description: "The identifier of the `send` operation confirmed as delivered by the recipient."
MessageSendErrorEvent:
type: object
description: "Event emitted when one or more segments of a message could not be dispatched to the network.
This indicates a network-level failure; the message was never fully transmitted."
fields:
requestId:
type: RequestId
description: "The identifier of the `send` operation that failed to dispatch."
error:
type: string
description: "Human-readable description of the dispatch failure."
MessageDeliveryErrorEvent:
type: object
description: "Event emitted when end-to-end delivery could not be confirmed.
The message reached the network and there's no need to explicit re-send.
Fired after `maxRetransmissions` attempts have been exhausted without receiving an SDS acknowledgement from the recipient."
fields:
requestId:
type: RequestId
description: "The identifier of the `send` operation that was not acknowledged by the recipient."
error:
type: string
description: "Human-readable description of the delivery failure."
RequestId:
type: string
description: "Unique identifier for a single `send` operation on a reliable channel.
It groups all segments produced by segmenting one message, so callers can correlate
acknowledgement and error events back to the original send call.
Internally, each segment is dispatched as an independent [MESSAGING-API](/standards/application/messaging-api.md) call,
producing one `RequestId` (as defined in [MESSAGING-API](/standards/application/messaging-api.md)) per segment.
A single `RequestId` therefore maps to one or more underlying [MESSAGING-API](/standards/application/messaging-api.md)'s `RequestId` values,
one per segment sent.
For example, the `RequestId` `Req_a` yields these MESSAGING-API requests:
`Req_a:1`, `Req_a:2`, ..., `Req_a:N`, where `Req_a:k` represents the k-th
MESSAGING-API segment `RequestId`.
That is, `Req_a` is the `RequestId` from the RELIABLE-CHANNEL-API spec PoV,
whereas `Req_a:k` is the `RequestId` from the MESSAGING-API spec PoV.
"
SegmentationConfig:
type: object
fields:
enableReedSolomon:
type: bool
default: false
description: When enabled, the message sender adds parity (redundant) segments to allow recovery in case of data segment loss. See [SEGMENTATION](./segmentation.md).
segmentSizeBytes:
type: uint
default: 102400 # 100 KiB
description: "Maximum segment size in bytes.
Messages larger than this value are split before SDS processing."
persistence:
type: Persistence
description: "Backend for persisting partial reassembly state across restarts.
Implementations MUST use this backend to store received segments until all segments of a message have arrived and can be reassembled.
Refer to [SEGMENTATION](./segmentation.md) for the full definition of what state must be persisted."
SdsConfig:
type: object
description: Scalable Data Sync config items.
fields:
persistence:
type: Persistence
description: "Backend for persisting the SDS local history. Implementations MAY support custom backends."
acknowledgementTimeoutMs:
type: uint
default: 5000
description: "Time in milliseconds to wait for acknowledgement before retransmitting."
maxRetransmissions:
type: uint
default: 5
description: "Maximum number of retransmission attempts before considering delivery failed."
causalHistorySize:
type: uint
default: 2
description: "Number of message IDs to consider in the causal history. With longer value, a stronger correctness is guaranteed but it requires higher bandwidth and memory."
RateLimitConfig:
type: object
description: Rate limiting configuration, containing RLN-specific attributes.
fields:
enabled:
type: bool
default: false
description: "Whether rate limiting is enforced. SHOULD be true when RLN is active."
epochPeriodSec:
type: uint
default: 600 # 10 minutes
description: "The epoch size used by the RLN relay, in milliseconds."
Encryption:
type: object
description: "Interface for a pluggable encryption mechanism.
When provided as a parameter to `createReliableChannel`, the API consumer MUST implement both encrypt and decrypt operations.
Implementations MAY use different signatures than those described below, as long as each operation accepts a byte array and returns a byte array."
fields:
encrypt:
type: function
description: "Encrypts a byte payload. Returns the encrypted payload."
parameters:
- name: content
type: array<byte>
returns:
type: result<array<byte>, error>
decrypt:
type: function
description: "Decrypts a byte payload. Returns the decrypted payload."
parameters:
- name: payload
type: array<byte>
returns:
type: result<array<byte>, error>
Persistence:
type: object
description: "Interface for a pluggable SDS persistence backend.
Implementations MUST provide all functions required to save and retrieve SDS state per channel. Implementations MUST also provide the persistence method of interest, e.g., SQLite, custom encrypted storage, etc.
Refer to the [SDS spec](https://lip.logos.co/ift-ts/raw/sds.html) for the full definition of what state must be persisted."
```
## Security/Privacy Considerations
- This API does not provide confidentiality by default. An `Encryption` implementation MUST be supplied when confidentiality is required.
- Segment metadata (message ID, segment index, total segments) is visible to network observers unless encrypted by the hook.
- SDS acknowledgement messages are sent over the same content topic and are subject to the same confidentiality concerns.
- Rate limiting compliance is required to avoid exclusion from the network by RLN-enforcing relays.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

220
standards/application/segmentation.md Normal file
View File

@ -0,0 +1,220 @@
---
title: Message Segmentation and Reconstruction
name: Message Segmentation and Reconstruction
tags: [segmentation]
version: 0.1
status: raw
---
## Abstract
This specification defines an application-layer protocol for **segmentation** and **reconstruction** of messages carried over a transport/delivery service with a message-size limitation, when the original payload exceeds said limitation.
Applications partition the payload into multiple transport messages and reconstruct the original on receipt,
even when segments arrive out of order or up to a **predefined percentage** of segments are lost.
The protocol optionally uses **Reed–Solomon** erasure coding for fault tolerance.
All messages are wrapped in a `SegmentMessageProto`, including those that fit in a single segment.
## Motivation
Many message transport and delivery protocols impose a maximum message size that restricts the size of application payloads.
For example, Waku Relay typically propagates messages up to **150 KB** as per [64/WAKU2-NETWORK - Message](https://rfc.vac.dev/waku/standards/core/64/network#message-size).
To support larger application payloads, a segmentation layer is required.
This specification enables larger messages by partitioning them into multiple envelopes and reconstructing them at the receiver.
Erasure-coded parity segments provide resilience against partial loss or reordering.
## Terminology
- **original payload**: the full application payload before segmentation.
- **data segment**: one of the partitioned chunks of the original message payload.
- **parity segment**: an erasure-coded segment derived from the set of data segments.
- **segment message**: a wire-message whose `payload` field carries a serialized `SegmentMessageProto`.
- **`segmentSize`**: configured maximum size in bytes of each data segment's `payload` chunk (before protobuf serialization).
The key words **"MUST"**, **"MUST NOT"**, **"REQUIRED"**, **"SHALL"**, **"SHALL NOT"**, **"SHOULD"**, **"SHOULD NOT"**, **"RECOMMENDED"**, **"NOT RECOMMENDED"**, **"MAY"**, and **"OPTIONAL"** in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
## Wire Format
Each segmented message is encoded as a `SegmentMessageProto` protobuf message:
```protobuf
syntax = "proto3";
message SegmentMessageProto {
// Keccak256(original payload), 32 bytes
bytes entire_message_hash = 1;
// Data segment indexing
uint32 data_segment_index = 2; // zero-indexed sequence number for data segments
uint32 data_segment_count = 3; // number of data segments (>= 1)
// Segment payload (data or parity shard)
bytes payload = 4;
// Parity segment indexing
uint32 parity_segment_index = 5; // zero-based sequence number for parity segments
uint32 parity_segment_count = 6; // number of parity segments
// Segment type
bool is_parity = 7; // true for parity segments, false (default) for data segments
}
```
**Field descriptions:**
- `entire_message_hash`: A 32-byte Keccak256 hash of the original complete payload, used to identify which segments belong together and verify reconstruction integrity.
- `data_segment_index`: Zero-indexed sequence number identifying this data segment's position (0, 1, 2, ..., data_segment_count - 1). Set only on data segments.
- `data_segment_count`: Total number of data segments the original message was split into. Set on every segment (data and parity).
- `payload`: The actual chunk of data or parity information for this segment.
- `parity_segment_index`: Zero-based sequence number for parity segments. Set only on parity segments.
- `parity_segment_count`: Total number of parity segments generated. Set on every segment (data and parity) when Reed–Solomon parity is used; `0` (default) otherwise.
- `is_parity`: Explicit segment type marker. `false` (default) for data segments; `true` for parity segments.
A message is either a **data segment** (when `is_parity == false`) or a **parity segment** (when `is_parity == true`).
### Validation
Receivers **MUST** enforce:
- `entire_message_hash.length == 32`
- `data_segment_count >= 1`
- `data_segment_count + parity_segment_count < maxTotalSegments`
- **Data segments** (`is_parity == false`):
`data_segment_index < data_segment_count`
- **Parity segments** (`is_parity == true`):
`parity_segment_count > 0` AND `parity_segment_index < parity_segment_count`
No other combinations are permitted.
A `SegmentMessageProto` with `data_segment_count == 1` and `data_segment_index == 0` is a valid single-segment data message: the `payload` field carries the entire original payload (see [Sending](#sending)).
## Segmentation
### Sending
To transmit a payload, the sender:
- **MUST** compute a 32-byte `entire_message_hash = Keccak256(original_payload)`.
- **MUST** split the payload into one or more **data segments**,
each of size up to `segmentSize` bytes.
A payload of size ≤ `segmentSize` produces a single data segment (`data_segment_count == 1`).
- **MUST** pad the last segment to `segmentSize` for Reed-Solomon erasure coding (only if Reed-Solomon coding is enabled)
- **MAY** use Reed–Solomon erasure coding at the predefined parity rate.
- **MUST** encode every segment as a `SegmentMessageProto` with:
- The `entire_message_hash`
- `data_segment_count` (total number of data segments, always set)
- When Reed–Solomon parity is used, `parity_segment_count` (total number of parity segments, set on every segment)
- For data segments: `is_parity = false`, `data_segment_index`
- For parity segments: `is_parity = true`, `parity_segment_index`
- The raw payload data
- Send each segment as an individual transport message according to the underlying transport service.
This yields a deterministic wire format: every transmitted payload is a `SegmentMessageProto`.
### Receiving
Upon receiving a segmented message, the receiver:
- **MUST** validate each segment according to [Wire Format → Validation](#validation).
- **MUST** cache received segments
- **MUST** attempt reconstruction once at least `data_segment_count` distinct segments (data and parity combined) have been received:
- If all data segments are present, concatenate their `payload` fields in `data_segment_index` order.
- Otherwise, recover the payload via Reed–Solomon decoding over the available data and parity segments.
- **MUST** verify `Keccak256(reconstructed_payload)` matches `entire_message_hash`.
On mismatch,
the message **MUST** be discarded and logged as invalid.
- Once verified,
the reconstructed payload **SHALL** be delivered to the application.
---
## Implementation Suggestions
### Reed–Solomon
Implementations that apply parity **SHALL** use fixed-size shards of length `segmentSize`.
The reference implementation uses **nim-leopard** (Leopard-RS) with a maximum of **256 total shards**.
### Storage / Persistence
Segments may be persisted (e.g., SQLite) and indexed by `entire_message_hash` and by sender. Sender may be authenticated, this is out of scope of this spec.
Implementations **SHOULD** support:
- Duplicate detection and idempotent saves
- Completion flags to prevent duplicate processing
- Timeout-based cleanup of incomplete reconstructions
- Per-sender quotas for stored bytes and concurrent reconstructions
### Configuration
- `segmentSize` — maximum size in bytes of each data segment's payload chunk (before protobuf serialization).
**REQUIRED** parameter, configurable by the client.
- `parityRate` — fraction of parity shards relative to data shards.
Configurable by the client. Defaults to **0.125** (12.5%).
- `maxTotalSegments` — maximum number of total shards (data + parity) per message.
Implementation-specific parameter, fixed. The reference implementation uses **256**.
**Reconstruction capability:**
With the predefined parity rate, reconstruction is possible if **all data segments** are received or if **any combination of data + parity** totals at least `data_segment_count` (i.e., up to the predefined percentage of loss tolerated).
**API simplicity:**
Libraries **SHOULD** require only `segmentSize` from the application for normal operation.
---
## Security Considerations
### Privacy
`entire_message_hash` enables correlation of segments that belong to the same original message but does not reveal content.
To prevent this correlation, applications **SHOULD** encrypt each segment after segmentation (see [Encryption](#encryption)).
Traffic analysis may still identify segmented flows.
### Encryption
This specification does not provide confidentiality.
Applications **SHOULD** encrypt each segment after segmentation
(i.e., encrypt the serialized `SegmentMessageProto` prior to transmission),
so that `entire_message_hash` and other identifying fields are not visible to observers.
### Integrity
Implementations **MUST** verify the Keccak256 hash post-reconstruction and discard on mismatch.
### Denial of Service
To mitigate resource exhaustion:
- Limit total concurrent reconstructions and aggregate buffered bytes
- When sender identity is available, apply the same two limits per sender
- Enforce timeouts and size caps
- Validate segment counts (≤ 256)
- Consider rate-limiting at the transport layer (for example, via [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/waku/standards/core/17/rln-relay) on Waku)
---
## Deployment Considerations
**Overhead:**
- Bandwidth overhead ≈ the predefined parity rate from parity (if enabled)
- Additional per-segment overhead ≤ **100 bytes** (protobuf + metadata)
**Network impact:**Ac
- Larger messages increase transport traffic and storage;
operators **SHOULD** consider policy limits
**Compatibility:**
- Nodes that do **not** implement this specification cannot reconstruct any messages.
---
## References
1. [10/WAKU2 – Waku](https://rfc.vac.dev/waku/standards/core/10/waku2)
2. [11/WAKU2-RELAY – Relay](https://rfc.vac.dev/waku/standards/core/11/relay)
3. [14/WAKU2-MESSAGE – Message](https://rfc.vac.dev/waku/standards/core/14/message)
4. [64/WAKU2-NETWORK](https://rfc.vac.dev/waku/standards/core/64/network#message-size)
5. [nim-leopard](https://github.com/status-im/nim-leopard) – Nim bindings for Leopard-RS (Reed–Solomon)
6. [Leopard-RS](https://github.com/catid/leopard) – Fast Reed–Solomon erasure coding library
7. [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt) – Key words for use in RFCs to Indicate Requirement Levels

322
standards/application/waku-api.md
View File

@ -1,322 +0,0 @@
---
title: WAKU-API
name: Waku API definition
category: Standards Track
status: raw
tags: [reliability, application, api, protocol composition]
editor: Oleksandr Kozlov <oleksandr@status.im>
contributors:
- Oleksandr Kozlov <oleksandr@status.im>
- Prem Chaitanya Prathi <prem@status.im>
- Franck Royer <franck@status.im>
---
## Table of contents
<!-- TOC -->
* [Table of contents](#table-of-contents)
* [Abstract](#abstract)
* [Motivation](#motivation)
* [Syntax](#syntax)
* [API design](#api-design)
* [IDL](#idl)
* [Primitive types and general guidelines](#primitive-types-and-general-guidelines)
* [Language mappings](#language-mappings)
* [Application](#application)
* [The Waku API](#the-waku-api)
* [Initialise Waku node](#initialise-waku-node)
* [Type definitions](#type-definitions)
* [Function definitions](#function-definitions)
* [Predefined values](#predefined-values)
* [Extended definitions](#extended-definitions)
* [The Validation API](#the-validation-api)
* [Security/Privacy Considerations](#securityprivacy-considerations)
* [Copyright](#copyright)
<!-- TOC -->
## Abstract
This document specifies an Application Programming Interface (API) that is RECOMMENDED for developers of the [WAKU2](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/10/waku2.md) clients to implement,
and for consumers to use as a single entry point to its functionalities.
This API defines the RECOMMENDED interface for leveraging Waku protocols to send and receive messages.
Application developers SHOULD use it to access capabilities for peer discovery, message routing, and peer-to-peer reliability.
TODO: This spec must be further extended to include connection health inspection, message sending, subscription and store hash queries.
## Motivation
The accessibility of Waku protocols is capped by the accessibility of their implementations, and hence API.
This RFC enables a concerted effort to draft an API that is simple and accessible, and provides an opinion on sane defaults.
The API defined in this document is an opinionated-by-purpose method to use the more agnostic [WAKU2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md) protocols.
## Syntax
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
## API design
### IDL
A custom Interface Definition Language (IDL) in YAML is used to define the Waku API.
Existing IDL Such as OpenAPI, AsyncAPI or WIT do not exactly fit the requirements for this API.
Hence, instead of having the reader learn a new IDL, we propose to use a simple IDL with self-describing syntax.
An alternative would be to choose a programming language. However, such choice may express unintended opinions on the API.
### Primitive types and general guidelines
- No `default` means that the value is mandatory, meaning a `default` value implies an optional parameter.
- Primitive types are `string`, `int`, `bool`, `enum` and `uint`
- Complex pre-defined types are:
- `object`: object and other nested types.
- `array`: iterable object containing values of all the same type.
- `result`: an enum type that either contains a value or void (success), or an error (failure); The error is left to the implementor.
- `error`: Left to the implementor on whether `error` types are `string` or `object` in the given language.
- Usage of `result` is RECOMMENDED, usage of exceptions is NOT RECOMMENDED, no matter the language.
TODO: Review whether to specify categories of errors.
### Language mappings
How the API definition should be translated to specific languages.
```yaml
language_mappings:
typescript:
naming_convention:
- functions: "camelCase"
- variables: "camelCase"
- types: "PascalCase"
nim:
naming_convention:
- functions: "camelCase"
- variables: "camelCase"
- types: "PascalCase"
```
### Application
This API is designed for generic use and ease across all programming languages, for `edge` and `core` type nodes.
## The Waku API
```yaml
api_version: "0.0.1"
library_name: "waku"
description: "Waku: a private and censorship-resistant message routing library."
```
### Initialise Waku node
#### Type definitions
```yaml
types:
WakuNode:
type: object
description: "A Waku node instance."
NodeConfig:
type: object
fields:
mode:
type: string
constraints: [ "edge", "core" ]
default: "core" # "edge" for mobile and browser devices.
description: "The mode of operation of the Waku node; 'edge' of the network: relies on other nodes for message routing; 'core' of the network: fully participate to message routing."
protocols_config:
type: ProtocolsConfig
default: TheWakuNetworkPreset
networking_config:
type: NetworkConfig
default: DefaultNetworkingConfig
eth_rpc_endpoints:
type: array<string>
description: "Eth/Web3 RPC endpoint URLs, only required when RLN is used for message validation; fail-over available by passing multiple URLs. Accepting an object for ETH RPC will be added at a later stage."
ProtocolsConfig:
type: object
fields:
entry_nodes:
type: array<string>
default: []
description: "Nodes to connect to; used for discovery bootstrapping and quick connectivity. enrtree and multiaddr formats are accepted. If not provided, node does not bootstrap to the network (local dev)."
static_store_nodes:
type: array<string>
default: []
# TODO: confirm behaviour at implementation time.
description: "The passed nodes are prioritised for store queries."
cluster_id:
type: uint
auto_sharding_config:
type: AutoShardingConfig
default: DefaultAutoShardingConfig
description: "The auto-sharding config, if sharding mode is `auto`"
message_validation:
type: MessageValidation
description: "If the default config for TWN is not used, then we still provide default configuration for message validation."
default: DefaultMessageValidation
NetworkingConfig:
type: object
fields:
listen_ipv4:
type: string
default: "0.0.0.0"
description: "The network IP address on which libp2p and discv5 listen for inbound connections. Not applicable for some environments such as the browser."
p2p_tcp_port:
type: uint
default: 60000
description: "The TCP port used for libp2p, relay, etc aka, general p2p message routing. Not applicable for some environments such as the browser."
discv5_udp_port:
type: uint
default: 9000
description: "The UDP port used for discv5. Not applicable for some environments such as the browser."
AutoShardingConfig:
type: object
fields:
num_shards_in_cluster:
type: uint
description: "The number of shards in the configured cluster; this is a globally agreed value for each cluster."
MessageValidation:
type: object
fields:
max_message_size:
type: string
default: "150 KiB"
description: "Maximum message size. Accepted units: KiB, KB, and B. e.g. 1024KiB; 1500 B; etc."
# For now, RLN is the only message validation available
rln_config:
type: RlnConfig
# If the default config for TWN is not used, then we do not apply RLN
default: none
RlnConfig:
type: object
fields:
contract_address:
type: string
description: "The address of the RLN contract exposes `root` and `getMerkleRoot` ABIs"
chain_id:
type: uint
description: "The chain id on which the RLN contract is deployed"
epoch_size_sec:
type: uint
description: "The epoch size to use for RLN, in seconds"
```
#### Function definitions
```yaml
functions:
createNode:
description: "Initialise a Waku node instance"
parameters:
- name: nodeConfig
type: NodeConfig
description: "The Waku node configuration."
returns:
type: result<WakuNode, error>
```
#### Predefined values
```yaml
values:
DefaultNetworkingConfig:
type: NetworkConfig
fields:
listen_ipv4: "0.0.0.0"
p2p_tcp_port: 60000
discv5_udp_port: 9000
TheWakuNetworkPreset:
type: ProtocolsConfig
fields:
entry_nodes: [ "enrtree://AIRVQ5DDA4FFWLRBCHJWUWOO6X6S4ZTZ5B667LQ6AJU6PEYDLRD5O@sandbox.waku.nodes.status.im" ]
# On TWN, we encourage the usage of discovered store nodes
static_store_nodes: []
cluster_id: 1
auto_sharding_config:
fields:
num_shards_in_cluster: 8
message_validation: TheWakuNetworkMessageValidation
TheWakuNetworkMessageValidation:
type: MessageValidation
fields:
max_message_size: "150 KiB"
rln_config:
fields:
contract_address: "0xB9cd878C90E49F797B4431fBF4fb333108CB90e6"
chain_id: 59141
epoch_size_sec: 600 # 10 minutes
# If not preset is used, autosharding on one cluster is applied by default
# This is a safe default that abstract shards (content topic shard derivation), and it enables scaling at a later stage
DefaultAutoShardingConfig:
type: AutoShardingConfig
fields:
num_shards_in_cluster: 1
# If no preset is used, we only apply a max size limit to messages
DefaultMessageValidation:
type: MessageValidation
fields:
max_message_size: "150 KiB"
rln_config: none
```
#### Extended definitions
**`mode`**:
If the `mode` set is `edge`, the initialised `WakuNode` SHOULD use:
- [LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) as client
- [FILTER](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/12/filter.md) as client
- [STORE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md) as client
- [METADATA](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/66/metadata.md) as client
- [PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/34/peer-exchange.md) as client
- [P2P-RELIABILITY](/standards/application/p2p-reliability.md)
If the `mode` set is `core`, the initialised `WakuNode` SHOULD use:
- [RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) as service node
- [FILTER](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/12/filter.md) as service node
- [PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/34/peer-exchange.md) as service node
- [STORE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md) as client
- [METADATA](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/66/metadata.md) as client and service node
- [P2P-RELIABILITY](/standards/application/p2p-reliability.md)
- [DISCV5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md)
- [PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/34/peer-exchange.md) as client and service node
- [RENDEZVOUS](https://github.com/waku-org/specs/blob/master/standards/core/rendezvous.md) as client and service node
`edge` mode SHOULD be used if node functions in resource restricted environment,
whereas `core` SHOULD be used if node has no strong hardware or bandwidth restrictions.
## The Validation API
[WAKU2-RLN-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/17/rln-relay.md) is currently the primary message validation mechanism in place.
Work is scheduled to specify a validate API to enable plug-in validation.
As part of this API, it will be expected that a validation object can be passed,
that would contain all validation parameters including RLN.
In the time being, parameters specific to RLN are accepted for the message validation.
RLN can also be disabled.
## Security/Privacy Considerations
See [WAKU2-ADVERSARIAL-MODELS](https://github.com/waku-org/specs/blob/master/informational/adversarial-models.md).
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

31
standards/core/mix.md
View File

@ -1,10 +1,10 @@
---
title: WAKU-MIX
name: Waku Mix
editor: Prem Chaitanya Prathi <premprathi@proton.me>
editor: Prem Chaitanya Prathi <prem@status.im>
contributors:
- Akshaya Mani <akshaya@status.im>
- Hanno Cornelius <hanno@status.im>
- Akshaya Mani <akshaya@status.im>
- Hanno Cornelius <hanno@status.im>
---
## Tags
@ -12,7 +12,8 @@ contributors:
`waku/core-protocol`
# Abstract
The document describes [libp2p mix](https://rfc.vac.dev/vac/raw/mix/) integration into waku.
The document describes [libp2p mix](https://lip.logos.co/ift-ts/raw/mix.html) integration into waku.
This integration provides higher anonymity for users publishing or querying for messages to/from the Waku network.
This document covers integration of mix with [lightpush](https://rfc.vac.dev/waku/standards/core/19/lightpush) and [store](https://rfc.vac.dev/waku/standards/core/13/store) protocols.
@ -38,6 +39,7 @@ This network of mix nodes SHALL relay mix messages anonymously to the recepient.
Anonymity of [Filter](https://rfc.vac.dev/waku/standards/core/12/filter) users is not addressed by this document.
## Terminology
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
@ -79,15 +81,18 @@ Resource-restricted/Edge nodes with short connection windows MUST _only_ act as
Each waku node that supports the `mix intermediary or exit role` SHOULD indicate the same in its discoverable [ENR](https://github.com/waku-org/specs/blob/master/standards/core/enr.md).
The following fields MUST be set as part of the discoverable ENR of a mix waku node:
- The `bit 5` in the [waku2 ENR key](https://github.com/waku-org/specs/blob/master/standards/core/enr.md#waku2-enr-key) is reserved to indicate `mix` support. This bit MUST be set to true to indicate `mix` support.
- A new field `mix-key` SHOULD be set to the `ed25519 public key` which is used for sphinx encryption.
Adding these fields to the ENR may cause the ENR size to cross the 300 byte limit especially in case a node supports multiple transports.
Adding these fields to the ENR may cause the ENR size to cross the 300 byte limit especially in case a node supports multiple transports.
This limitation will have to be addressed in future.
### Discovery
Mix protocol provides better anonymity when a sender node has a sufficiently large pool of mix nodes to do path selection.
This moves the problem into discovery domain and requires the following from discovery mechanisms:
1. It is important for nodes to be able to discover as many nodes as possible quickly. This becomes especially important for edge nodes that come online just to publish/query messages for a short period of time.
2. The discovery mechanism MUST be unbiased and not biased toward specific subsets (e.g., nodes that are topologically closer).
3. It is important to have the most recent online status of the nodes so that mix paths that are selected are not broken which lead to reliability issues.
@ -101,15 +106,16 @@ Mix protocol in waku network SHOULD have `rate-limiting/spam` protection to hand
1. Any node can generate a mix packet and publish into the mix network. Hence there needs to be some validation as to who is allowed to publish and whether the user is within allowed rate-limits.
2. Any node can intentionally generate paths which are broken and send messages into the mix network.
3. An attacker can spawn a huge number of mix nodes so that user behaviour is observed in order to determine traffic patterns and deanonymize users.
3. An attacker can spawn a huge number of mix nodes so that user behaviour is observed in order to determine traffic patterns and deanonymize users.
There is a need to enforce rate-limits and spam protect the mix network.
There is a need to enforce rate-limits and spam protect the mix network.
The rate-limiting and spam protection shall be addressed as part of future work.
## Tradeoffs
Using `mix` protocol for publishing and querying messages adds certain overhead which is primarily the delay in delivering message to the destination.
The overall additional delay `D` depends on the following params:
- path length `L`
- delay added by each intermediary node `dm`
- connection establishment time `dc`
@ -129,8 +135,9 @@ Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [libp2p mix](https://rfc.vac.dev/vac/raw/mix/)
- [waku lightpush](https://rfc.vac.dev/waku/standards/core/19/lightpush)
- [waku relay](https://rfc.vac.dev/waku/standards/core/11/relay)
- [ENR](https://github.com/waku-org/specs/blob/master/standards/core/enr.md)
- [sphinx encryption](https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf)
- [libp2p mix](https://lip.logos.co/ift-ts/raw/mix.html)
- [waku lightpush](https://rfc.vac.dev/waku/standards/core/19/lightpush)
- [waku relay](https://rfc.vac.dev/waku/standards/core/11/relay)
- [ENR](https://github.com/waku-org/specs/blob/master/standards/core/enr.md)
- [sphinx encryption](https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf)

133
standards/core/rendezvous.md
View File

@ -1,66 +1,121 @@
---
title: WAKU-RENDEZVOUS
name: Waku Rendezvous discovery
editor: Simon-Pierre Vivier <simvivier@status.im>
contributors:
editor: Prem Chaitanya Prathi <premprathi@proton.me>
contributors: Simon-Pierre Vivier <simvivier@status.im>
---
## Abstract
This document describes the goal,
strategy and usage of the libp2p rendezvous protocol by Waku.
strategy, usage, and changes to the libp2p rendezvous protocol by Waku.
Rendezvous is one of the discovery methods that can be used by Waku.
It supplements Discovery v5 and Waku peer exchange.
## Background and Rationale
How do new nodes join the network is the question that discovery answers.
Discovery must be fast, pertinent and resilient to attacks.
Rendezvous is both fast and allow the discovery of relevant peers,
although it design can be easily abused
due to it's lack of protection against denial of service atacks.
The properties of rendezvous complements well the slower but safer methods like Discv5.
To contribute well, a Waku node must know a sufficient number of peers with
a wide variety of capabilities.
By using rendezvous in combination with
[Discv5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md#node-discovery-protocol-v5) and
[34/WAKU2-PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/34/peer-exchange.md#abstract),
Waku nodes will reach a good number of meaningful peers
faster than through a single discovery method.
Waku needs discovery mechanism(s) that are both rapid and robust against attacks.
Fully centralised discovery (such as DNS lookup) may be fast but is not secure.
Fully decentralised discovery (such as discv5) may be robust, but too slow for some bootstrapping use cases
Rendezvous provides a limited, balanced solution that trades off some robustness for speed.
It's meant to complement not replaced fully decentralised discovery mechanisms, like discv5
By combining rendezvous with
[Discv5](https://github.com/ethereum/devp2p/blob/bc76b9809a30e6dc5c8dcda996273f0f9bcf7108/discv5/discv5.md#node-discovery-protocol-v5) and
[34/WAKU2-PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/dd397adc594c121ce3e10b7e81b5c2ed4818c0a6/waku/standards/core/34/peer-exchange.md#abstract),
Waku nodes can more quickly reach a meaningful set of peers
than by relying on a single discovery method.
## Semantics
Waku rendezvous fully inherits the [libp2p rendezvous semantics](https://github.com/libp2p/specs/blob/master/rendezvous/README.md#rendezvous-protocol).
Waku rendezvous extends the [libp2p rendezvous semantics](https://github.com/libp2p/specs/blob/69c4fdf5da3a07d2f392df6a892c07256c1885c0/rendezvous/README.md#rendezvous-protocol) by using `WakuPeerRecord` instead of the standard libp2p `PeerRecord`.
This allows nodes to advertise additional Waku-specific metadata beyond what is available in the standard libp2p peer record.
## Specifications
The namespaces used to register and request MUST be in the format `rs/cluster-id/shard`.
Refer to [RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md) for cluster and shard information.
Every [Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) node SHOULD be initialized as a rendezvous point.
**Libp2p Protocol identifier**: `/vac/waku/rendezvous/1.0.0`
### Wire Protocol
Nodes advertise their information through `WakuPeerRecord`, a custom peer record structure designed for Waku rendezvous.
Since this is a customPeerRecord, we define a private multicodec value of `0x300000` as per [multicodec table](https://github.com/multiformats/multicodec/blob/0c6c7d75f1580af329847dbc9900859a445ed980/table.csv).
The `WakuPeerRecord` is defined as follows:
**WakuPeerRecord fields:**
- `peer_id`: The libp2p PeerId of the node.
- `seqNo`: The time at which the record was created or last updated (Unix epoch, seconds).
- `multiaddrs`: A list of multiaddresses for connectivity.
- `mix_public_key`: The Mix protocol public key (only present for nodes supporting Mix).
**Encoding:**
WakuPeerRecord is encoded as a protobuf message. The exact schema is:
```protobuf
message WakuPeerRecord {
string peer_id = 1;
uint64 seqNo = 2;
repeated string multiaddrs = 3;
optional bytes mix_public_key = 4;
}
```
When a node discovers peers through rendezvous, it receives the complete `WakuPeerRecord` for each peer, allowing it to make informed decisions about which peers to connect to based on their advertised information.
### Namespace Format
The [rendezvous namespaces](https://github.com/libp2p/specs/blob/69c4fdf5da3a07d2f392df6a892c07256c1885c0/rendezvous/README.md#the-protocol) used to register and request peer records
MUST be in the format `rs/<cluster-id>/<capability>`.
`<capability>` is a string representing the individual capability for which a discoverable Waku peer record is registered.
The Waku peer record is separately registered against each capability for which discovery is desired.
The only defined capability for now is `mix`, representing [Waku Mix](https://github.com/waku-org/specs/blob/fe5cfdf823085b8c9560df81b38bc342d6b1cb38/standards/core/mix.md) support.
For example, a Waku peer record for a node supporting mix protocol in cluster `1` will be registered against a namespace: `rs/1/mix`.
This allows for discovering peers with specific capabilities within a given cluster.
Currently, this is used for Mix protocol discovery where the capability field specifies `mix`.
Refer to [RELAY-SHARDING](https://github.com/waku-org/specs/blob/fe5cfdf823085b8c9560df81b38bc342d6b1cb38/standards/core/relay-sharding.md) for cluster information.
### Registration and Discovery
Every [Waku Relay](https://github.com/vacp2p/rfc-index/blob/dd397adc594c121ce3e10b7e81b5c2ed4818c0a6/waku/standards/core/11/relay.md) node SHOULD be initialized as a rendezvous point.
Each relay node that participates in discovery
MUST register with random rendezvous points at regular intervals.
The RECOMMENDED registration interval is 10 seconds.
We RECOMMEND a registration interval of 10 seconds.
The node SHOULD register once for every shard it supports,
registering only the namespace corresponding to that shard.
All relay nodes participating in rendezvous discovery SHOULD advertise their information using `WakuPeerRecord`. For nodes supporting the Mix protocol, the `mix_public_key` field MUST be included.
All advertised records MUST conform to the `WakuPeerRecord` definition.
We RECOMMEND that rendezvous points expire registrations after 1 minute,
in order to keep discovered peer records to those recentrly online.
It is RECOMMENDED that rendezvous points expire registrations after 1 minute (60 seconds TTL)
to keep discovered peer records limited to those recently online.
At startup, every Waku node SHOULD discover peers by
sending requests to random rendezvous points,
once for each shard it supports.
At startup, every Waku node supporting Mix SHOULD discover peers by
sending requests to random rendezvous points for the Mix capability namespace.
We RECOMMEND a maximum of 12 peers will be requested each time.
This number is enough for good GossipSub connectivity and
minimize the load on rendezvous points.
It is RECOMMENDED a maximum of 12 peers be requested each time.
This number is sufficient for good GossipSub connectivity and
minimizes the load on rendezvous points.
We RECOMMEND that bootstrap nodes participate in rendezvous discovery and
that other discovery methods are used in conjunction and
### Operational Recommendations
It is RECOMMENDED that bootstrap nodes participate in rendezvous discovery and
that other discovery methods are used in conjunction and
continue discovering peers for the lifetime of the local node.
For resource-constrained devices or light clients, a client-only mode MAY be used
where nodes only query for peers without acting as rendezvous points themselves
and without advertising their own peer records.
## Future Work
Namespaces will not contain capabilities yet but may in the future. If the need arise nodes could use rendezvous to discover peers with specific capabilities.
The protocol currently supports advertising Mix-specific capabilities (Mix public keys) through `WakuPeerRecord`.
Future enhancements could include:
- Extending `WakuPeerRecord` to advertise other Waku protocol capabilities (Relay, Store, Filter, Lightpush, etc.)
- Supporting shard-based namespaces (e.g., `rs/<cluster-id>/<shard>`) for general relay peer discovery without capability filtering
- Batch registration support allowing nodes to register across multiple namespaces in a single request
# Copyright
@ -68,7 +123,9 @@ Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
# References
- [Discv5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md#node-discovery-protocol-v5)
- [34/WAKU2-PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/34/peer-exchange.md#abstract)
- [Libp2p Rendezvous](https://github.com/libp2p/specs/blob/master/rendezvous/README.md#rendezvous-protocol)
- [Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [Discv5](https://github.com/ethereum/devp2p/blob/bc76b9809a30e6dc5c8dcda996273f0f9bcf7108/discv5/discv5.md#node-discovery-protocol-v5)
- [ENR](https://github.com/ethereum/devp2p/blob/bc76b9809a30e6dc5c8dcda996273f0f9bcf7108/enr.md)
- [34/WAKU2-PEER-EXCHANGE](https://github.com/vacp2p/rfc-index/blob/dd397adc594c121ce3e10b7e81b5c2ed4818c0a6/waku/standards/core/34/peer-exchange.md#abstract)
- [Libp2p Rendezvous](https://github.com/libp2p/specs/blob/69c4fdf5da3a07d2f392df6a892c07256c1885c0/rendezvous/README.md#rendezvous-protocol)
- [Relay](https://github.com/vacp2p/rfc-index/blob/dd397adc594c121ce3e10b7e81b5c2ed4818c0a6/waku/standards/core/11/relay.md)