"use strict";(function(){const t={cache:!0};t.doc={id:"id",field:["title","content"],store:["title","href","section"]};const e=FlexSearch.create("balance",t);window.bookSearchIndex=e,e.add({id:0,href:"/spec/1/",title:"1/COSS",section:"Docs",content:`This document describes a consensus-oriented specification system (COSS) for building interoperable technical specifications. COSS is based on a lightweight editorial process that seeks to engage the widest possible range of interested parties and move rapidly to consensus through working code.
This specification is based on Unprotocols 2/COSS, used by the ZeromMQ project. It is equivalent except for some areas:
recommending the use of a permissive licenses, such as CC0 (with the exception of this document); miscellaneous metadata, editor, and format/link updates; more inheritance from the [IETF Standards Process][https://www.rfc-editor.org/rfc/rfc2026.txt], e.g. using RFC categories: Standards Track, Informational, and Best Common Practice; standards track specifications SHOULD follow a specific structure that both streamlines editing, and helps implementers to quickly comprehend the specification specifications MUST feature a header providing specific meta information License # Copyright (c) 2008-22 the Editor and Contributors.
This Specification is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This Specification is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, see http://www.gnu.org/licenses.
Change Process # This document is governed by the 1/COSS (COSS).
Language # The key words \u0026ldquo;MUST\u0026rdquo;, \u0026ldquo;MUST NOT\u0026rdquo;, \u0026ldquo;REQUIRED\u0026rdquo;, \u0026ldquo;SHALL\u0026rdquo;, \u0026ldquo;SHALL NOT\u0026rdquo;, \u0026ldquo;SHOULD\u0026rdquo;, \u0026ldquo;SHOULD NOT\u0026rdquo;, \u0026ldquo;RECOMMENDED\u0026rdquo;, \u0026ldquo;MAY\u0026rdquo;, and \u0026ldquo;OPTIONAL\u0026rdquo; in this document are to be interpreted as described in RFC 2119.
Goals # The primary goal of COSS is to facilitate the process of writing, proving, and improving new technical specifications. A \u0026ldquo;technical specification\u0026rdquo; defines a protocol, a process, an API, a use of language, a methodology, or any other aspect of a technical environment that can usefully be documented for the purposes of technical or social interoperability.
COSS is intended to above all be economical and rapid, so that it is useful to small teams with little time to spend on more formal processes.
Principles:
We aim for rough consensus and running code; inspired by the IETF Tao. Specifications are small pieces, made by small teams. Specifications should have a clearly responsible editor. The process should be visible, objective, and accessible to anyone. The process should clearly separate experiments from solutions. The process should allow deprecation of old specifications. Specifications should take minutes to explain, hours to design, days to write, weeks to prove, months to become mature, and years to replace.
Specifications have no special status except that accorded by the community.
Architecture # COSS is designed around fast, easy to use communications tools. Primarily, COSS uses a wiki model for editing and publishing specifications texts.
The domain is the conservancy for a set of specifications in a certain area. Each domain is implemented as an Internet domain, hosting a wiki and optionally other communications tools. Each specification is a set of wiki pages, together with comments, attached files, and other resources. Important specifications may also exist as subdomains, i.e. child wikis. Individuals can become members of the domain by completing the necessary legal clearance. The copyright, patent, and trademark policies of the domain must be clarified in an Intellectual Property policy that applies to the domain.
Specifications exist as multiple pages, one page per version of the specification (see \u0026ldquo;Branching and Merging\u0026rdquo;, below), which may be assigned URIs that include an incremental number. Thus, we refer to a specification by specifying its domain, number, and short name. New versions of the same specification will have new numbers. The syntax for a specification reference is:
\u0026lt;domain\u0026gt;/spec/\u0026lt;number\u0026gt;/\u0026lt;shortname\u0026gt; For example, this specification is rfc.vac.dev/spec/1/COSS. The short form 1/COSS may be used when referring to the specification from other specifications in the same domain.
Every specification (including branches) carries a different number.
COSS Lifecycle # Every specification has an independent lifecycle that documents clearly its current status.
A specification has six possible states that reflect its maturity and contractual weight:
Raw Specifications # All new specifications are raw specifications. Changes to raw specifications can be unilateral and arbitrary. Those seeking to implement a raw specification should ask for it to be made a draft specification. Raw specifications have no contractual weight.
Draft Specifications # When raw specifications can be demonstrated, they become draft specifications. Changes to draft specifications should be done in consultation with users. Draft specifications are contracts between the editors and implementers.
Stable Specifications # When draft specifications are used by third parties, they become stable specifications. Changes to stable specifications should be restricted to cosmetic ones, errata and clarifications. Stable specifications are contracts between editors, implementers, and end-users.
Deprecated Specifications # When stable specifications are replaced by newer draft specifications, they become deprecated specifications. Deprecated specifications should not be changed except to indicate their replacements, if any. Deprecated specifications are contracts between editors, implementers and end-users.
Retired Specifications # When deprecated specifications are no longer used in products, they become retired specifications. Retired specifications are part of the historical record. They should not be changed except to indicate their replacements, if any. Retired specifications have no contractual weight.
Deleted Specifications # Deleted specifications are those that have not reached maturity (stable) and were discarded. They should not be used and are only kept for their historical value. Only Raw and Draft specifications can be deleted.
Editorial control # A specification MUST have a single responsible editor, the only person who SHALL change the status of the specification through the lifecycle stages.
A specification MAY also have additional contributors who contribute changes to it. It is RECOMMENDED to use a process similar to C4 process to maximize the scale and diversity of contributions.
Unlike the original C4 process however, it is RECOMMENDED to use CC0 as a more permissive license alternative. We SHOULD NOT use GPL or GPL-like license. One exception is this specification, as this was the original license for this specification.
The editor is responsible for accurately maintaining the state of specifications and for handling all comments on the specification.
Branching and Merging # Any member of the domain MAY branch a specification at any point. This is done by copying the existing text, and creating a new specification with the same name and content, but a new number. The ability to branch a specification is necessary in these circumstances:
To change the responsible editor for a specification, with or without the cooperation of the current responsible editor. To rejuvenate a specification that is stable but needs functional changes. This is the proper way to make a new version of a specification that is in stable or deprecated status. To resolve disputes between different technical opinions. The responsible editor of a branched specification is the person who makes the branch.
Branches, including added contributions, are derived works and thus licensed under the same terms as the original specification. This means that contributors are guaranteed the right to merge changes made in branches back into their original specifications.
Technically speaking, a branch is a different specification, even if it carries the same name. Branches have no special status except that accorded by the community.
Conflict resolution # COSS resolves natural conflicts between teams and vendors by allowing anyone to define a new specification. There is no editorial control process except that practised by the editor of a new specification. The administrators of a domain (moderators) may choose to interfere in editorial conflicts, and may suspend or ban individuals for behaviour they consider inappropriate.
Specification Structure # Meta Information # Specifications MUST contain the following metadata. It is RECOMMENDED that specification metadata is specified as a YAML header (where possible). This will enable programmatic access to specification metadata.
Key Value Type Example shortname short name string 1/COSS title full name string Consensus-Oriented Specification System status status string draft category category string Best Current Practice tags 0 or several tags list waku-application, waku-core-protocol editor editor name/email string Oskar Thoren oskar@status.im contributors contributors list - Pieter Hintjens ph@imatix.com - André Rebentisch andre@openstandards.de - Alberto Barrionuevo abarrio@opentia.es - Chris Puttick chris.puttick@thehumanjourney.net - Yurii Rashkovskii yrashk@gmail.com Specification Template # Standards Track specifications SHOULD be based on the Vac RFC template.
Conventions # Where possible editors and contributors are encouraged to:
Refer to and build on existing work when possible, especially IETF specifications. Contribute to existing specifications rather than reinvent their own. Use collaborative branching and merging as a tool for experimentation. Use Semantic Line Breaks: https://sembr.org/. Appendix A. Color Coding # It is RECOMMENDED to use color coding to indicate specification\u0026rsquo;s status. Color coded specifications SHOULD use the following color scheme:
`}),e.add({id:1,href:"/spec/10/",title:"10/WAKU2",section:"Docs",content:` Abstract # Waku v2 is family of modular peer-to-peer protocols for secure communication. The protocols are designed to be secure, privacy-preserving, censorship-resistant and being able to run in resource restricted environments. At a high level, it implements Pub/Sub over libp2p and adds a set of capabilities to it. These capabilities are things such as: (i) retrieving historical messages for mostly-offline devices (ii) adaptive nodes, allowing for heterogeneous nodes to contribute to the network (iii) preserving bandwidth usage for resource-restriced devices
This makes Waku ideal for running a p2p protocol on mobile and in similarly restricted environments.
Historically, it has its roots in Waku v1, which stems from Whisper, originally part of the Ethereum stack. However, Waku v2 acts more as a thin wrapper for PubSub and has a different API. It is implemented in an iterative manner where initial focus is on porting essential functionality to libp2p. See rough road map (2020) for more historical context.
Motivation and goals # Waku as a family of protocols is designed to have a set of properties that are useful for many applications:
Useful for generalized messaging. Many applications require some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix. Waku is designed to work for all these scenarios.
Peer-to-peer. Applications sometimes have requirements that make them suitable for peer-to-peer solutions:
Censorship-resistant with no single point of failure Adaptive and scalable network Shared infrastructure Runs anywhere. Applications often run in restricted environments, where resources or the environment is restricted in some fashion. For example:
Limited bandwidth, CPU, memory, disk, battery, etc Not being publicly connectable Only being intermittently connected; mostly-offline Privacy-preserving. Applications often have a desire for some privacy guarantees, such as:
Pseudonymity and not being tied to any personally identifiable information (PII) Metadata protection in transit Various forms of unlinkability, etc Modular design. Applications often have different trade-offs when it comes to what properties they and their users value. Waku is designed in a modular fashion where an application protocol or node can choose what protocols they run. We call this concept adaptive nodes.
For example:
Resource usage vs metadata protection Providing useful services to the network vs mostly using it Stronger guarantees for spam protection vs economic registration cost For more on the concept of adaptive nodes and what this means in practice, please see the 30/ADAPTIVE-NODES spec.
Network interaction domains # While Waku is best thought of as a single cohesive thing, there are three network interaction domains:
(a) gossip domain (b) discovery domain (c) req/resp domain
Protocols and identifiers # Since Waku v2 is built on top of libp2p, many protocols have a libp2p protocol identifier. The current main protocol identifiers are:
/vac/waku/relay/2.0.0 /vac/waku/store/2.0.0-beta4 /vac/waku/filter/2.0.0-beta1 /vac/waku/lightpush/2.0.0-beta1 This is in addition to protocols that specify messages, payloads, and recommended usages. Since these aren\u0026rsquo;t negotiated libp2p protocols, they are referred to by their RFC ID. For example:
14/WAKU2-MESSAGEand 26/WAKU2-PAYLOAD for message payloads 23/WAKU2-TOPICS and 27/WAKU2-PEERS for recommendations around usage There are also more experimental libp2p protocols such as:
/vac/waku/swap/2.0.0-beta1 /vac/waku/waku-rln-relay/2.0.0-alpha1 These protocols and their semantics are elaborated on in their own specs.
Use of libp2p and protobuf # Unless otherwise specified, all protocols are implemented over libp2p and use Protobuf by default. Since messages are exchanged over a bi-directional binary stream, as a convention, libp2p protocols prefix binary message payloads with the length of the message in bytes. This length integer is encoded as a protobuf varint.
Gossip domain # Waku is using gossiping to disseminate messages throughout the network.
Protocol identifier: /vac/waku/relay/2.0.0
See 11/WAKU2-RELAY spec for more details.
For an experimental privacy-preserving economic spam protection mechanism, see 17/WAKU2-RLN-RELAY.
See 23/WAKU2-TOPICS for more information about recommended topic usage.
Direct use of libp2p protocols # In addition to /vac/waku/* protocols, Waku v2 MAY directly use the following libp2p protocols:
libp2p ping protocol with protocol id /ipfs/ping/1.0.0 for liveness checks between peers, or to keep peer-to-peer connections alive.
libp2p identity and identity/push with protocol IDs /ipfs/id/1.0.0 and
/ipfs/id/push/1.0.0 respectively, as basic means for capability discovery. These protocols are anyway used by the libp2p connection establishment layer Waku v2 is built on. We plan to introduce a new Vac capability discovery protocol with better anonymity properties and more functionality.
Transports # Waku v2 is built in top of libp2p, and like libp2p it strives to be transport agnostic. We define a set of recommended transports in order to achieve a baseline of interoperability between clients.
This section describes these recommended transports.
Waku client implementations SHOULD support the TCP transport.
Where TCP is supported it MUST be enabled for both dialing and listening, even if other transports are available.
Waku v2 nodes where the environment do not allow to use TCP directly, MAY use other transports.
A Waku v2 node SHOULD support secure websockets for bidirectional communication streams, for example in a web browser context.
A node MAY support unsecure websockets if required by the application or running environment.
Discovery domain # Discovery methods # Waku v2 can retrieve a list of nodes to connect to using DNS-based discovery as per EIP-1459. While this is a useful way of bootstrapping connection to a set of peers, it MAY be used in conjunction with an ambient peer discovery procedure to find still other nodes to connect to, such as Node Discovery v5. More ambient peer discovery methods are being tested for Waku v2, and will be specified for wider adoption. It is possible to bypass the discovery domain by specifying static nodes.
Use of ENR # 31/WAKU2-ENR describes the usage of EIP-778 ENR (Ethereum Node Records) for Waku v2 discovery purposes. It introduces two new ENR fields, multiaddrs and waku2, that a Waku v2 node MAY use for discovery purposes. These fields MUST be used under certain conditions, as set out in the spec. Both EIP-1459 DNS-based discovery and Node Discovery v5 operates on ENR, and it\u0026rsquo;s reasonable to expect even wider utility for ENR in Waku v2 networks in future.
Request/Reply domain # In addition to the Gossip domain, Waku provides a set of Request/Reply protocols. They are primarily used in order to get Waku to run in resource restricted environments, such as low bandwidth or being mostly offline.
Historical message support # Protocol identifier*: /vac/waku/store/2.0.0-beta4
This is used to fetch historical messages for mostly offline devices. See 13/WAKU2-STORE spec for more details.
There is also an experimental fault-tolerant addition to the store protocol that relaxes the high availability requirement. See 21/WAKU2-FT-STORE
Content filtering # Protocol identifier*: /vac/waku/filter/2.0.0-beta1
This is used to make fetching of a subset of messages more bandwidth preserving. See 12/WAKU2-FILTER spec for more details.
Light push # Protocol identifier*: /vac/waku/lightpush/2.0.0-beta1
This is used for nodes with short connection windows and limited bandwidth to publish messages into the Waku network. See 19/WAKU2-LIGHTPUSH spec for more details.
Other protocols # The above is a non-exhaustive list, and due to the modular design of Waku there may be other protocols here that provide a useful service to the Waku network.
Overview of protocol interaction # See the sequence diagram below for an overview of how different protocols interact.
We have six nodes, A-F. The protocols initially mounted are indicated as such. The PubSub topics pubtopic1 and pubtopic2 is used for routing and indicates that it is subscribed to messages on that topic for relay, see 11/WAKU2-RELAY for details. Ditto for 13/WAKU2-STORE where it indicates that these messages are persisted on that node.
Node A creates a WakuMessage msg1 with a ContentTopic contentTopic1. See 14/WAKU2-MESSAGE for more details. If WakuMessage version is set to 1, we use the 6/WAKU1 compatible data field with encryption. See 7/WAKU-DATA for more details.
Node F requests to get messages filtered by PubSub topic pubtopic1 and ContentTopic contentTopic1. Node D subscribes F to this filter and will in the future forward messages that match that filter. See 12/WAKU2-FILTER for more details.
Node A publishes msg1 on pubtopic1 and subscribes to that relay topic pick it up. It then gets relayed further from B to D, but not C since it doesn\u0026rsquo;t subscribe to that topic. See 11/WAKU2-RELAY.
Node D saves msg1 for possible later retrieval by other nodes. See 13/WAKU2-STORE.
Node D also pushes msg1 to F, as it has previously subscribed F to this filter. See 12/WAKU2-FILTER.
At a later time, Node E comes online. It then requests messages matching pubtopic1 and contentTopic1 from Node D. Node D responds with messages meeting this (and possibly other) criteria. See 13/WAKU2-STORE.
Appendix A: Upgradability and Compatibility # Compatibility with Waku v1 # Waku v1 and Waku v2 are different protocols all together. They use a different transport protocol underneath; Waku v1 is devp2p RLPx based while Waku v2 uses libp2p. The protocols themselves also differ as does their data format. Compatibility can be achieved only by using a bridge that not only talks both devp2p RLPx and libp2p, but that also transfers (partially) the content of a packet from one version to the other.
See 15/WAKU-BRIDGE for details on a bidirectional bridge mode.
Appendix B: Security # Each protocol layer of Waku v2 provides a distinct service and is associated with a separate set of security features and concerns. Therefore, the overall security of Waku v2 depends on how the different layers are utilized. In this section, we overview the security properties of Waku v2 protocols against a static adversarial model which is described below. Note that a more detailed security analysis of each Waku protocol is supplied in its respective specification as well.
Primary Adversarial Model # In the primary adversarial model, we consider adversary as a passive entity that attempts to collect information from others to conduct an attack, but it does so without violating protocol definitions and instructions.
The following are not considered as part of the adversarial model:
An adversary with a global view of all the peers and their connections. An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). Specifically, the communication channels are assumed to be secure. Security Features # Pseudonymity # Waku v2 by default guarantees pseudonymity for all of the protocol layers since parties do not have to disclose their true identity and instead they utilize libp2p PeerID as their identifiers. While pseudonymity is an appealing security feature, it does not guarantee full anonymity since the actions taken under the same pseudonym i.e., PeerID can be linked together and potentially result in the re-identification of the true actor.
Anonymity / Unlinkability # At a high level, anonymity is the inability of an adversary in linking an actor to its data/performed action (the actor and action are context-dependent). To be precise about linkability, we use the term Personally Identifiable Information (PII) to refer to any piece of data that could potentially be used to uniquely identify a party. For example, the signature verification key, and the hash of one\u0026rsquo;s static IP address are unique for each user and hence count as PII. Notice that users\u0026rsquo; actions can be traced through their PIIs (e.g., signatures) and hence result in their re-identification risk. As such, we seek anonymity by avoiding linkability between actions and the actors / actors\u0026rsquo; PII. Concerning anonymity, Waku v2 provides the following features:
Publisher-Message Unlinkability: This feature signifies the unlinkability of a publisher to its published messages in the 11/WAKU2-RELAY protocol. The Publisher-Message Unlinkability is enforced through the StrictNoSign policy due to which the data fields of pubsub messages that count as PII for the publisher must be left unspecified.
Subscriber-Topic Unlinkability: This feature stands for the unlinkability of the subscriber to its subscribed topics in the 11/WAKU2-RELAY protocol. The Subscriber-Topic Unlinkability is achieved through the utilization of a single PubSub topic. As such, subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID. This level of unlinkability / anonymity is known as k-anonymity where k is proportional to the system size (number of subscribers). Note that there is no hard limit on the number of the pubsub topics, however, the use of one topic is recommended for the sake of anonymity.
Spam protection # This property indicates that no adversary can flood the system (i.e., publishing a large number of messages in a short amount of time), either accidentally or deliberately, with any kind of message i.e. even if the message content is valid or useful. Spam protection is partly provided in 11/WAKU2-RELAY through the scoring mechanism provided for by GossipSub v1.1. At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
Data confidentiality, Integrity, and Authenticity # Confidentiality can be addressed through data encryption whereas integrity and authenticity are achievable through digital signatures. These features are provided for in 14/WAKU2-MESSAGE (version 1)\` through payload encryption as well as encrypted signatures.
Security Considerations # Lack of anonymity/unlinkability in the protocols involving direct connections including 13/WAKU2-STORE and 12/WAKU2-FILTER protocols: The anonymity/unlinkability is not guaranteed in the protocols like 13/WAKU2-STORE and 12/WAKU2-FILTER where peers need to have direct connections to benefit from the designated service. This is because during the direct connections peers utilize PeerID to identify each other, therefore the service obtained in the protocol is linkable to the beneficiary\u0026rsquo;s PeerID (which counts as PII). For 13/WAKU2-STORE, the queried node would be able to link the querying node\u0026rsquo;s PeerID to its queried topics. Likewise, in the 12/WAKU2-FILTER, a full node can link the light node\u0026rsquo;s PeerIDs to its content filter.
Appendix C: Implementation Notes # Implementation Matrix # There are multiple implementations of Waku v2 and its protocols:
nim-waku (Nim) go-waku (Go) js-waku (NodeJS and Browser) Below you can find an overview of the specs that they implement as they relate to Waku v2. This includes Waku v1 specs, as they are used for bridging between the two networks.
Spec nim-waku (Nim) go-waku (Go) js-waku (Node JS) js-waku (Browser JS) 6/WAKU1 ✔ 7/WAKU-DATA ✔ ✔ 8/WAKU-MAIL ✔ 9/WAKU-RPC ✔ 10/WAKU2 ✔ 🚧 🚧 🚧 11/WAKU2-RELAY ✔ ✔ ✔ ✔ 12/WAKU2-FILTER ✔ ✔ 13/WAKU2-STORE ✔ ✔ ✔* ✔* 14/WAKU2-MESSAGE ✔ ✔ ✔ ✔ 15/WAKU2-BRIDGE ✔ 16/WAKU2-RPC ✔ 17/WAKU2-RLN-RELAY 🚧 18/WAKU2-SWAP 🚧 19/WAKU2-LIGHTPUSH ✔ ✔ ✔** ✔** 21/WAKU2-FAULT-TOLERANT-STORE ✔ ✔ *js-waku implements 13/WAKU2-STORE as a querying node only. **js-waku only implements 19/WAKU2-LIGHTPUSH requests.
Recommendations for clients # To implement a minimal Waku v2 client, we recommend implementing the following subset in the following order:
10/WAKU2 - this spec 11/WAKU2-RELAY - for basic operation 14/WAKU2-MESSAGE - version 0 (unencrypted) 13/WAKU2-STORE - for historical messaging (query mode only) To get compatibility with Waku v1:
7/WAKU-DATA 14/WAKU2-MESSAGE - version 1 (encrypted with 7/WAKU-DATA) For an interoperable keep-alive mechanism:
libp2p ping protocol, with periodic pings to connected peers Appendix D: Future work # The following features are currently experimental and under research and initial implementation:
Economic Spam resistance: We aim to enable an incentivized spam protection technique to enhance 11/WAKU2-RELAY by using rate limiting nullifiers. More details on this can be found in 17/WAKU2-RLN-RELAY. In this advanced method, peers are limited to a certain rate of messaging per epoch and an immediate financial penalty is enforced for spammers who break this rate.
Prevention of Denial of Service (DoS) and Node Incentivization: Denial of service signifies the case where an adversarial node exhausts another node\u0026rsquo;s service capacity (e.g., by making a large number of requests) and makes it unavailable to the rest of the system. DoS attack is to be mitigated through the accounting model as described in 18/WAKU2-SWAP. In a nutshell, peers have to pay for the service they obtain from each other. In addition to incentivizing the service provider, accounting also makes DoS attacks costly for malicious peers. The accounting model can be used in 13/WAKU2-STORE and 12/WAKU2-FILTER to protect against DoS attacks.
Additionally, this gives node operators who provide a useful service to the network an incentive to perform that service. See 18/WAKU2-SWAP for more details on this piece of work.
Copyright # Copyright and related rights waived via CC0.
References # libp2p specs
6/WAKU1 spec
Whisper spec (EIP627)
Waku v2 plan
Protocol Identifiers
Protobuf varint encoding
7/WAKU-DATA spec
11/WAKU2-RELAY spec
13/WAKU2-STORE spec
12/WAKU2-FILTER spec
15/WAKU2-BRIDGE spec
k-anonymity
GossipSub v1.1
14/WAKU2-MESSAGE spec
17/WAKU2-RLN-RELAY spec
18/WAKU2-SWAP spec
Ping protocol
EIP-1459
Ambient peer discovery
Node Discovery v5
31/WAKU2-ENR spec
EIP-778
`}),e.add({id:2,href:"/spec/11/",title:"11/WAKU2-RELAY",section:"Docs",content:`11/WAKU2-RELAY specifies a Publish/Subscribe approach to peer-to-peer messaging with a strong focus on privacy, censorship-resistance, security and scalability. Its current implementation is a minor extension of the libp2p GossipSub protocol and prescribes gossip-based dissemination. As such the scope is limited to defining a separate protocol id for 11/WAKU2-RELAY, establishing privacy and security requirements, and defining how the underlying GossipSub is to be interpreted and implemented within the Waku and cryptoeconomic domain. 11/WAKU2-RELAY should not be confused with libp2p circuit relay.
Protocol identifier: /vac/waku/relay/2.0.0
Security Requirements # The 11/WAKU2-RELAY protocol is designed to provide the following security properties under a static Adversarial Model. Note that data confidentiality, integrity, and authenticity are currently considered out of scope for 11/WAKU2-RELAY and must be handled by higher layer protocols such as 14/WAKU2-MESSAGE.
Publisher-Message Unlinkability: This property indicates that no adversarial entity can link a published Message to its publisher. This feature also implies the unlinkability of the publisher to its published topic ID as the Message embodies the topic IDs.
Subscriber-Topic Unlinkability: This feature stands for the inability of any adversarial entity from linking a subscriber to its subscribed topic IDs.
Terminology # Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one\u0026rsquo;s static IP address are unique for each user and hence count as PII.
Adversarial Model # Any entity running the 11/WAKU2-RELAY protocol is considered an adversary. This includes publishers, subscribers, and all the peers\u0026rsquo; direct connections. Furthermore, we consider the adversary as a passive entity that attempts to collect information from others to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious subscriber hides the messages it receives from other subscribers as it is against the description of 11/WAKU2-RELAY. However, a malicious subscriber may learn which topics are subscribed to by which peers. The following are not considered as part of the adversarial model: An adversary with a global view of all the peers and their connections. An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). In other words, the communication channels are assumed to be secure. Wire Specification # The PubSub interface specification defines the protobuf RPC messages exchanged between peers participating in a GossipSub network. We republish these messages here for ease of reference and define how 11/WAKU2-RELAY uses and interprets each field.
Protobuf definitions # The PubSub RPC messages are specified using protocol buffers v2
syntax = \u0026#34;proto2\u0026#34;; message RPC { repeated SubOpts subscriptions = 1; repeated Message publish = 2; message SubOpts { optional bool subscribe = 1; optional string topicid = 2; } message Message { optional string from = 1; optional bytes data = 2; optional bytes seqno = 3; repeated string topicIDs = 4; optional bytes signature = 5; optional bytes key = 6; } } NOTE: The various control messages defined for GossipSub are used as specified there.
NOTE: The TopicDescriptor is not currently used by 11/WAKU2-RELAY.
Message fields # The Message protobuf defines the format in which content is relayed between peers. 11/WAKU2-RELAY specifies the following usage requirements for each field:
The from field MUST NOT be used, following the StrictNoSign signature policy.
The data field MUST be filled out with a WakuMessage. See 14/WAKU2-MESSAGE for more details.
The seqno field MUST NOT be used, following the StrictNoSign signature policy.
The topicIDs field MUST contain the topics that a message is being published on.
The signature field MUST NOT be used, following the StrictNoSign signature policy.
The key field MUST NOT be used, following the StrictNoSign signature policy.
SubOpts fields # The SubOpts protobuf defines the format in which subscription options are relayed between peers. A 11/WAKU2-RELAY node MAY decide to subscribe or unsubscribe from topics by sending updates using SubOpts. The following usage requirements apply:
The subscribe field MUST contain a boolean, where true indicates subscribe and false indicates unsubscribe to a topic.
The topicid field MUST contain the topic.
Signature Policy # The StrictNoSign option MUST be used, to ensure that messages are built without the signature, key, from and seqno fields. Note that this does not merely imply that these fields be empty, but that they MUST be absent from the marshalled message.
Security Analysis # Publisher-Message Unlinkability: To address publisher-message unlinkability, one should remove any PII from the published message. As such, 11/WAKU2-RELAY follows the StrictNoSign policy as described in libp2p PubSub specs. As the result of the StrictNoSign policy, Messages should be built without the from, signature and key fields since each of these three fields individually counts as PII for the author of the message (one can link the creation of the message with libp2p peerId and thus indirectly with the IP address of the publisher). Note that removing identifiable information from messages cannot lead to perfect unlinkability. The direct connections of a publisher might be able to figure out which Messages belong to that publisher by analyzing its traffic. The possibility of such inference may get higher when the data field is also not encrypted by the upper-level protocols. Subscriber-Topic Unlinkability: To preserve subscriber-topic unlinkability, it is recommended by 10/WAKU2 to use a single PubSub topic in the 11/WAKU2-RELAY protocol. This allows an immediate subscriber-topic unlinkability where subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID. This level of unlinkability / anonymity is known as k-anonymity where k is proportional to the system size (number of participants of Waku relay protocol). However, note that 11/WAKU2-RELAY supports the use of more than one topic. In case that more than one topic id is utilized, preserving unlinkability is the responsibility of the upper-level protocols which MAY adopt partitioned topics technique to achieve K-anonymity for the subscribed peers.
Future work # Economic spam resistance: In the spam-protected 11/WAKU2-RELAY protocol, no adversary can flood the system with spam messages (i.e., publishing a large number of messages in a short amount of time). Spam protection is partly provided by GossipSub v1.1 through scoring mechanism. At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score. 11/WAKU2-RELAY aims at enabling an advanced spam protection mechanism with economic disincentives by utilizing Rate Limiting Nullifiers. In a nutshell, peers must conform to a certain message publishing rate per a system-defined epoch, otherwise, they get financially penalized for exceeding the rate. More details on this new technique can be found in 17/WAKU2-RLN-RELAY.
Providing Unlinkability, Integrity and Authenticity simultaneously: Integrity and authenticity are typically addressed through digital signatures and Message Authentication Code (MAC) schemes, however, the usage of digital signatures (where each signature is bound to a particular peer) contradicts with the unlinkability requirement (messages signed under a certain signature key are verifiable by a verification key that is bound to a particular publisher). As such, integrity and authenticity are missing features in 11/WAKU2-RELAY in the interest of unlinkability. In future work, advanced signature schemes like group signatures can be utilized to enable authenticity, integrity, and unlinkability simultaneously. In a group signature scheme, a member of a group can anonymously sign a message on behalf of the group as such the true signer is indistinguishable from other group members. Copyright # Copyright and related rights waived via CC0.
References # 10/WAKU2
14/WAKU2-MESSAGE
17/WAKU-RLN
GossipSub v1.0
GossipSub v1.1
K-anonimity
libp2p concepts: Publish/Subscribe
libp2p protocol negotiation
Partitioned topics
Protocol Buffers
PubSub interface for libp2p (r2, 2019-02-01)
Waku v1 spec
Whisper spec (EIP627)
`}),e.add({id:3,href:"/spec/12/",title:"12/WAKU2-FILTER",section:"Docs",content:`WakuFilter is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of WakuRelay specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
Content filtering # Protocol identifier*: /vac/waku/filter/2.0.0-beta1
Content filtering is a way to do message-based filtering. Currently the only content filter being applied is on contentTopic. This corresponds to topics in Waku v1.
Rationale # Unlike the store protocol for historical messages, this protocol allows for native lower latency scenarios such as instant messaging. It is thus complementary to it.
Strictly speaking, it is not just doing basic request response, but performs sender push based on receiver intent. While this can be seen as a form of light pub/sub, it is only used between two nodes in a direct fashion. Unlike the Gossip domain, this is meant for light nodes which put a premium on bandwidth. No gossiping takes place.
It is worth noting that a light node could get by with only using the store protocol to query for a recent time window, provided it is acceptable to do frequent polling.
Design Requirements # The effectiveness and reliability of the content filtering service enabled by WakuFilter protocol rely on the high availability of the full nodes as the service providers. To this end, full nodes must feature high uptime (to persistently listen and capture the network messages) as well as high Bandwidth (to provide timely message delivery to the light nodes).
Security Consideration # Note that while using WakuFilter allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the WakuFilter, however, potential solutions in this regard are sketched below in Future Work section.
Terminology # The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one\u0026rsquo;s static IP address are unique for each user and hence count as PII.
Adversarial Model # Any node running the WakuFilter protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one\u0026rsquo;s subscribed content filter as it is against the description of the WakuFilter protocol.
The following are not considered as part of the adversarial model:
An adversary with a global view of all the nodes and their connections. An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure. Protobuf # message FilterRequest { bool subscribe = 1; string topic = 2; repeated ContentFilter contentFilters = 3; message ContentFilter { string contentTopic = 1; } } message MessagePush { repeated WakuMessage messages = 1; } message FilterRPC { string requestId = 1; FilterRequest request = 2; MessagePush push = 3; } FilterRPC # A node MUST send all Filter messages (FilterRequest, MessagePush) wrapped inside a FilterRPC this allows the node handler to determine how to handle a message as the Waku Filter protocol is not a request response based protocol but instead a push based system.
The requestId MUST be a uniquely generated string. When a MessagePush is sent the requestId MUST match the requestId of the subscribing FilterRequest whose filters matched the message causing it to be pushed.
FilterRequest # A FilterRequest contains an optional topic, zero or more content filters and a boolean signifying whether to subscribe or unsubscribe to the given filters. True signifies \u0026lsquo;subscribe\u0026rsquo; and false signifies \u0026lsquo;unsubscribe\u0026rsquo;.
A node that sends the RPC with a filter request and subscribe set to \u0026rsquo;true\u0026rsquo; requests that the filter node SHOULD notify the light requesting node of messages matching this filter.
A node that sends the RPC with a filter request and subscribe set to \u0026lsquo;false\u0026rsquo; requests that the filter node SHOULD stop notifying the light requesting node of messages matching this filter if it is currently doing so.
The filter matches when content filter and, optionally, a topic is matched. Content filter is matched when a WakuMessage contentTopic field is the same.
A filter node SHOULD honor this request, though it MAY choose not to do so. If it chooses not to do so it MAY tell the light why. The mechanism for doing this is currently not specified. For notifying the light node a filter node sends a MessagePush message.
Since such a filter node is doing extra work for a light node, it MAY also account for usage and be selective in how much service it provides. This mechanism is currently planned but underspecified.
MessagePush # A filter node that has received a filter request SHOULD push all messages that match this filter to a light node. These WakuMessage\u0026rsquo;s are likely to come from the relay protocol and be kept at the Node, but there MAY be other sources or protocols where this comes from. This is up to the consumer of the protocol.
A filter node MUST NOT send a push message for messages that have not been requested via a FilterRequest.
If a specific light node isn\u0026rsquo;t connected to a filter node for some specific period of time (e.g. a TTL), then the filter node MAY choose to not push these messages to the node. This period is up to the consumer of the protocol and node implementation, though a reasonable default is one minute.
Future Work # Anonymous filter subscription: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the WakuFilter protocol would be able to link nodes to their subscribed content filers. The current version of the WakuFilter protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor. Note that the current structure of filter requests i.e., FilterRPC does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity. By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are Oblivious Transfers and one-way Private Set Intersections (PSI). Changelog # Next # Added initial threat model and security analysis. 2.0.0-beta2 # Initial draft version. Released 2020-10-28
Fix: Ensure contentFilter is a repeated field, on implementation Change: Add ability to unsubscribe from filters. Make subscribe an explicit boolean indication. Edit protobuf field order to be consistent with libp2p. 2.0.0-beta1 # Initial draft version. Released 2020-10-05
Copyright # Copyright and related rights waived via CC0.
References # Message Filtering (Wikipedia)
Libp2p PubSub spec - topic validation
`}),e.add({id:4,href:"/spec/12/",title:"12/WAKU2-FILTER",section:"Docs",content:`previous versions: 00
WakuFilter is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of WakuRelay specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
Content filtering # Protocol identifiers:
filter-subscribe: /vac/waku/filter-subscribe/2.0.0-beta1 filter-push: /vac/waku/filter-push/2.0.0-beta1 Content filtering is a way to do message-based filtering. Currently the only content filter being applied is on contentTopic. This corresponds to topics in Waku v1.
Rationale # Unlike the store protocol for historical messages, this protocol allows for native lower latency scenarios such as instant messaging. It is thus complementary to it.
Strictly speaking, it is not just doing basic request response, but performs sender push based on receiver intent. While this can be seen as a form of light pub/sub, it is only used between two nodes in a direct fashion. Unlike the Gossip domain, this is meant for light nodes which put a premium on bandwidth. No gossiping takes place.
It is worth noting that a light node could get by with only using the store protocol to query for a recent time window, provided it is acceptable to do frequent polling.
Design Requirements # The effectiveness and reliability of the content filtering service enabled by WakuFilter protocol rely on the high availability of the full nodes as the service providers. To this end, full nodes must feature high uptime (to persistently listen and capture the network messages) as well as high Bandwidth (to provide timely message delivery to the light nodes).
Security Consideration # Note that while using WakuFilter allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the WakuFilter, however, potential solutions in this regard are sketched below in Future Work section.
Terminology # The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one\u0026rsquo;s static IP address are unique for each user and hence count as PII.
Adversarial Model # Any node running the WakuFilter protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one\u0026rsquo;s subscribed content filter as it is against the description of the WakuFilter protocol.
The following are not considered as part of the adversarial model:
An adversary with a global view of all the nodes and their connections. An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure. Protobuf # syntax = \u0026#34;proto3\u0026#34;; // 12/WAKU2-FILTER rfc: https://rfc.vac.dev/spec/12/ package waku.filter.v2; // Protocol identifier: /vac/waku/filter-subscribe/2.0.0-beta1 message FilterSubscribeRequest { enum FilterSubscribeType { SUBSCRIBER_PING = 0; SUBSCRIBE = 1; UNSUBSCRIBE = 2; UNSUBSCRIBE_ALL = 3; } string request_id = 1; FilterSubscribeType filter_subscribe_type = 2; // Filter criteria optional string pubsub_topic = 10; repeated string content_topics = 11; } message FilterSubscribeResponse { string request_id = 1; uint32 status_code = 10; optional string status_desc = 11; } // Protocol identifier: /vac/waku/filter-push/2.0.0-beta1 message MessagePush { WakuMessage waku_message = 1; optional string pubsub_topic = 2; } Filter-Subscribe # A filter service node MUST support the filter-subscribe protocol to allow filter clients to subscribe, modify, refresh and unsubscribe a desired set of filter criteria. The combination of different filter criteria for a specific filter client node is termed a \u0026ldquo;subscription\u0026rdquo;. A filter client is interested in receiving messages matching the filter criteria in its registered subscriptions.
Since a filter service node is consuming resources to provide this service, it MAY account for usage and adapt its service provision to certain clients. An incentive mechanism is currently planned but underspecified.
Filter Subscribe Request # A client node MUST send all filter requests in a FilterSubscribeRequest message. This request MUST contain a request_id. The request_id MUST be a uniquely generated string. Each request MUST include a filter_subscribe_type, indicating the type of request.
Filter Subscribe Response # In return to any FilterSubscribeRequest, a filter service node SHOULD respond with a FilterSubscribeResponse with a requestId matching that of the request. This response MUST contain a status_code indicating if the request was successful or not. Successful status codes are in the 2xx range. Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed. In addition, the filter service node MAY choose to provide a more detailed status description in the status_desc field.
Filter matching # In the description of each request type below, the term \u0026ldquo;filter criteria\u0026rdquo; refers to the combination of pubsub_topic and a set of content_topics. The request MAY include filter criteria, conditional to the selected filter_subscribe_type. If the request contains filter criteria, it MUST contain a pubsub_topic and the content_topics set MUST NOT be empty. A WakuMessage matches filter criteria when its content_topic is in the content_topics set and it was published on a matching pubsub_topic.
Filter Subscribe Types # The following filter subscribe types are defined:
SUBSCRIBER_PING # A filter client that sends a FilterSubscribeRequest with filter_subscribe_type set to SUBSCRIBER_PING requests that the service node SHOULD indicate if it has any active subscriptions for this client. The filter client SHOULD exclude any filter criteria from the request. The filter service node SHOULD respond with a success code if it has any active subscriptions for this client or an error code if not. The filter service node SHOULD ignore any filter criteria in the request.
SUBSCRIBE # A filter client that sends a FilterSubscribeRequest with filter_subscribe_type set to SUBSCRIBE requests that the service node SHOULD push messages matching this filter to the client. The filter client MUST include the desired filter criteria in the request. A client MAY use this request type to modify an existing subscription by providing additional filter criteria in a new request. A client MAY use this request type to refresh an existing subscription by providing the same filter criteria in a new request. The filter service node SHOULD respond with a success code if it successfully honored this request or an error code if not. The filter service node SHOULD respond with an error code and discard the request if the subscribe request does not contain valid filter criteria, i.e. both a pubsub_topic and a non-empty content_topics set.
UNSUBSCRIBE # A filter client that sends a FilterSubscribeRequest with filter_subscribe_type set to UNSUBSCRIBE requests that the service node SHOULD stop pushing messages matching this filter to the client. The filter client MUST include the filter criteria it desires to unsubscribe from in the request. A client MAY use this request type to modify an existing subscription by providing a subset of the original filter criteria to unsubscribe from in a new request. The filter service node SHOULD respond with a success code if it successfully honored this request or an error code if not. The filter service node SHOULD respond with an error code and discard the request if the unsubscribe request does not contain valid filter criteria, i.e. both a pubsub_topic and a non-empty content_topics set.
UNSUBSCRIBE_ALL # A filter client that sends a FilterSubscribeRequest with filter_subscribe_type set to UNSUBSCRIBE_ALL requests that the service node SHOULD stop pushing messages matching any filter to the client. The filter client SHOULD exclude any filter criteria from the request. The filter service node SHOULD remove any existing subscriptions for this client. It SHOULD respond with a success code if it successfully honored this request or an error code if not.
Filter-Push # A filter client node MUST support the filter-push protocol to allow filter service nodes to push messages matching registered subscriptions to this client.
A filter service node SHOULD push all messages matching the filter criteria in a registered subscription to the subscribed filter client. These WakuMessages are likely to come from 11/WAKU2-RELAY, but there MAY be other sources or protocols where this comes from. This is up to the consumer of the protocol.
If a message push fails, the filter service node MAY consider the client node to be unreachable. If a specific filter client node is not reachable from the service node for a period of time, the filter service node MAY choose to stop pushing messages to the client and remove its subscription. This period is up to the service node implementation. We consider 1 minute to be a reasonable default.
Message Push # Each message MUST be pushed in a MessagePush message. Each MessagePush MUST contain one (and only one) waku_message. If this message was received on a specific pubsub_topic, it SHOULD be included in the MessagePush. A filter client SHOULD NOT respond to a MessagePush. Since the filter protocol does not include caching or fault-tolerance, this is a best effort push service with no bundling or guaranteed retransmission of messages. A filter client SHOULD verify that each MessagePush it receives originated from a service node where the client has an active subscription and that it matches filter criteria belonging to that subscription.
Future Work # Anonymous filter subscription: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the WakuFilter protocol would be able to link nodes to their subscribed content filers. The current version of the WakuFilter protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor. Note that the current structure of filter requests i.e., FilterRPC does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity. By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are Oblivious Transfers and one-way Private Set Intersections (PSI). Changelog # Next # Added initial threat model and security analysis. 2.0.0-beta2 # Initial draft version. Released 2020-10-28
Fix: Ensure contentFilter is a repeated field, on implementation Change: Add ability to unsubscribe from filters. Make subscribe an explicit boolean indication. Edit protobuf field order to be consistent with libp2p. 2.0.0-beta1 # Initial draft version. Released 2020-10-05
Copyright # Copyright and related rights waived via CC0.
References # Message Filtering (Wikipedia)
Libp2p PubSub spec - topic validation
`}),e.add({id:5,href:"/spec/13/",title:"13/WAKU2-STORE",section:"Docs",content:`This specification explains the Waku 13/WAKU2-STORE protocol which enables querying of messages received through relay protocol and stored by other nodes. It also supports pagination for more efficient querying of historical messages.
Protocol identifier*: /vac/waku/store/2.0.0-beta4
Design Requirements # Nodes willing to provide storage service using 13/WAKU2-STORE protocol SHOULD provide a complete and full view of message history. As such, they are required to be highly available and in specific have a high uptime to consistently receive and store network messages. The high uptime requirement makes sure that no message is missed out hence a complete and intact view of the message history is delivered to the querying nodes. Nevertheless, in case that storage provider nodes cannot afford high availability, the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
The concept of \u0026ldquo;ephemeral\u0026rdquo; messages introduced in [14/WAKU2-MESSAGE](/spec/14) affects 13/WAKU2-STORE as well. Nodes running 13/WAKU2-STORE SHOULD support \u0026ldquo;ephemeral\u0026rdquo; messages as specified in 14/WAKU2-MESSAGE. Nodes running 13/WAKU2-STORE SHOULD NOT store messages with the ephemeral flag set to true.
Security Consideration # The main security consideration to take into account while using 13/WAKU2-STORE is that a querying node have to reveal their content filters of interest to the queried node, hence potentially compromising their privacy.
Terminology # The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one\u0026rsquo;s static IP address are unique for each user and hence count as PII.
Adversarial Model # Any peer running the 13/WAKU2-STORE protocol, i.e. both the querying node and the queried node, are considered as an adversary. Furthermore, we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but it does so without violating protocol definitions and instructions. As we evolve the protocol, further adversarial models will be considered. For example, under the passive adversarial model, no malicious node hides or lies about the history of messages as it is against the description of the 13/WAKU2-STORE protocol.
The following are not considered as part of the adversarial model:
An adversary with a global view of all the peers and their connections. An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure. Wire Specification # Peers communicate with each other using a request / response API. The messages sent are Protobuf RPC messages which are implemented using protocol buffers v3. The followings are the specifications of the Protobuf messages.
Payloads # syntax = \u0026#34;proto3\u0026#34;; message Index { bytes digest = 1; sint64 receiverTime = 2; sint64 senderTime = 3; string pubsubTopic = 4; } message PagingInfo { uint64 pageSize = 1; Index cursor = 2; enum Direction { BACKWARD = 0; FORWARD = 1; } Direction direction = 3; } message ContentFilter { string contentTopic = 1; } message HistoryQuery { // the first field is reserved for future use string pubsubtopic = 2; repeated ContentFilter contentFilters = 3; PagingInfo pagingInfo = 4; } message HistoryResponse { // the first field is reserved for future use repeated WakuMessage messages = 2; PagingInfo pagingInfo = 3; enum Error { NONE = 0; INVALID_CURSOR = 1; } Error error = 4; } message HistoryRPC { string request_id = 1; HistoryQuery query = 2; HistoryResponse response = 3; } Index # To perform pagination, each WakuMessage stored at a node running the 13/WAKU2-STORE protocol is associated with a unique Index that encapsulates the following parts.
digest: a sequence of bytes representing the SHA256 hash of a WakuMessage. The hash is computed over the concatenation of contentTopic and payload fields of a WakuMessage (see 14/WAKU2-MESSAGE). receiverTime: the UNIX time in nanoseconds at which the waku message is received by the receiving node. senderTime: the UNIX time in nanoseconds at which the waku message is generated by its sender. pubsubTopic: the pubsub topic on which the waku message is received PagingInfo # PagingInfo holds the information required for pagination. It consists of the following components.
pageSize: A positive integer indicating the number of queried WakuMessages in a HistoryQuery (or retrieved WakuMessages in a HistoryResponse). cursor: holds the Index of a WakuMessage. direction: indicates the direction of paging which can be either FORWARD or BACKWARD. ContentFilter # ContentFilter carries the information required for filtering historical messages.
contentTopic represents the content topic of the queried historical Waku messages. This field maps to the contentTopic field of the 14/WAKU2-MESSAGE. HistoryQuery # RPC call to query historical messages.
The pubsubTopic field MUST indicate the pubsub topic of the historical messages to be retrieved. This field denotes the pubsub topic on which waku messages are published. This field maps to topicIDs field of Message in 11/WAKU2-RELAY. Leaving this field empty means no filter on the pubsub topic of message history is requested. This field SHOULD be left empty in order to retrieve the historical waku messages regardless of the pubsub topics on which they are published. The contentFilters field MUST indicate the list of content filters based on which the historical messages are to be retrieved. Leaving this field empty means no filter on the content topic of message history is required. This field SHOULD be left empty in order to retrieve historical waku messages regardless of their content topics. PagingInfo holds the information required for pagination.
Its pageSize field indicates the number of WakuMessages to be included in the corresponding HistoryResponse. It is RECOMMENDED that the queried node defines a maximum page size internally. If the querying node leaves the pageSize unspecified, or if the pageSize exceeds the maximum page size, the queried node SHOULD auto-paginate the HistoryResponse to no more than the configured maximum page size. This allows mitigation of long response time for HistoryQuery. In the forward pagination request, the messages field of the HistoryResponse shall contain at maximum the pageSize amount of waku messages whose Index values are larger than the given cursor (and vise versa for the backward pagination). Note that the cursor of a HistoryQuery may be empty (e.g., for the initial query), as such, and depending on whether the direction is BACKWARD or FORWARD the last or the first pageSize waku messages shall be returned, respectively. Sorting Messages # The queried node MUST sort the WakuMessages based on their Index, where the senderTime constitutes the most significant part and the digest comes next, and then perform pagination on the sorted result. As such, the retrieved page contains an ordered list of WakuMessages from the oldest message to the most recent one. Alternatively, the receiverTime (instead of senderTime ) MAY be used to sort WakuMessages during the paging process. However, we RECOMMEND the use of the senderTime for sorting as it is invariant and consistent across all the nodes. This has the benefit of cursor reusability i.e., a cursor obtained from one node can be consistently used to query from another node. However, this cursor reusability does not hold when the receiverTime is utilized as the receiver time is affected by the network delay and nodes\u0026rsquo; clock asynchrony.
HistoryResponse # RPC call to respond to a HistoryQuery call.
The messages field MUST contain the messages found, these are [WakuMessage] types as defined in the corresponding specification. PagingInfo holds the paging information based on which the querying node can resume its further history queries. The pageSize indicates the number of returned Waku messages (i.e., the number of messages included in the messages field of HistoryResponse). The direction is the same direction as in the corresponding HistoryQuery. In the forward pagination, the cursor holds the Index of the last message in the HistoryResponse messages (and the first message in the backward paging). Regardless of the paging direction, the retrieved messages are always sorted in ascending order based on their timestamp as explained in the sorting messages section, that is, from the oldest to the most recent. The requester shall embed the returned cursor inside its next HistoryQuery to retrieve the next page of the Waku messages.
The cursor obtained from one node SHOULD NOT be used in a request to another node because the result MAY be different. The error field contains information about any error that has occurred while processing the corresponding HistoryQuery. NONE stands for no error. This is also the default value. INVALID_CURSOR means that the cursor field of HistoryQuery does not match with the Index of any of the WakuMessages persisted by the queried node. Future Work # Anonymous query: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e., without disclosing the exact topics of waku messages they are interested in.
As such, no adversary in the 13/WAKU2-STORE protocol would be able to learn which peer is interested in which content filters i.e., content topics of waku message. The current version of the 13/WAKU2-STORE protocol does not provide anonymity for historical queries as the querying node needs to directly connect to another node in the 13/WAKU2-STORE protocol and explicitly disclose the content filters of its interest to retrieve the corresponding messages. However, one can consider preserving anonymity through one of the following ways: By hiding the source of the request i.e., anonymous communication. That is the querying node shall hide all its PII in its history request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor. Note that the current structure of historical requests does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve query anonymity. By deploying secure 2-party computations in which the querying node obtains the historical messages of a certain topic whereas the queried node learns nothing about the query. Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI). Robust and verifiable timestamps: Messages timestamp is a way to show that the message existed prior to some point in time. However, the lack of timestamp verifiability can create room for a range of attacks, including injecting messages with invalid timestamps pointing to the far future.
To better understand the attack, consider a store node whose current clock shows 2021-01-01 00:00:30 (and assume all the other nodes have a synchronized clocks +-20seconds). The store node already has a list of messages (m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20) that are sorted based on their timestamp.
An attacker sends a message with an arbitrary large timestamp e.g., 10 hours ahead of the correct clock (m',2021-01-01 10:00:30). The store node places m' at the end of the list (m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m',2021-01-01 10:00:30). Now another message arrives with a valid timestamp e.g., (m11, 2021-01-01 00:00:45). However, since its timestamp precedes the malicious message m', it gets placed before m' in the list i.e., (m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30). In fact, for the next 10 hours, m' will always be considered as the most recent message and served as the last message to the querying nodes irrespective of how many other messages arrive afterward.
A robust and verifiable timestamp allows the receiver of a message to verify that a message has been generated prior to the claimed timestamp. One solution is the use of open timestamps e.g., block height in Blockchain-based timestamps. That is, messages contain the most recent block height perceived by their senders at the time of message generation. This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or seconds (e.g., in Ethereum 2.0) from the time of origination.
References # Open timestamps Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:6,href:"/spec/14/",title:"14/WAKU2-MESSAGE",section:"Docs",content:` Abstract # Waku v2 is a family of modular peer-to-peer protocols for secure communication. These protocols are designed to be secure, privacy-preserving, and censorship-resistant and can run in resource-restricted environments. At a high level, Waku v2 implements a Pub/Sub messaging pattern over libp2p and adds capabilities.
The present document specifies the Waku v2 message format, a way to encapsulate the messages sent with specific information security goals, and Whisper/Waku v1 backward compatibility.
Motivation # When sending messages over Waku, there are multiple requirements:
One may have a separate encryption layer as part of the application. One may want to provide efficient routing for resource-restricted devices. One may want to provide compatibility with Waku v1 envelopes. One may want encrypted payloads by default. One may want to provide unlinkability to get metadata protection. This specification attempts to provide for these various requirements.
Semantics # Waku Message # A Waku message is constituted by the combination of data payload and attributes that, for example, a publisher sends to a topic and is eventually delivered to subscribers.
Waku message attributes are key-value pairs of metadata associated with a message. And the message data payload is the part of the transmitted Waku message that is the actual message information. The data payload is also treated as a Waku message attribute for convenience.
Message Attributes # The payload attribute MUST contain the message data payload to be sent.
The content_topic attribute MUST specify a string identifier that can be used for content-based filtering.
The meta attribute, if present, contains an arbitrary application-specific variable-length byte array with a maximum length limit of 64 bytes. This attribute can be utilized to convey supplementary details to various Waku protocols, thereby enabling customized processing based on its contents.
The version attribute, if present, contains a version number to discriminate different types of payload encryption. If omitted, the value SHOULD be interpreted as version 0.
The timestamp attribute, if present, signifies the time at which the message was generated by its sender. This attribute MAY contain the Unix epoch time in nanoseconds. If the attribute is omitted, it SHOULD be interpreted as timestamp 0.
The ephemeral attribute, if present, signifies the transient nature of the message. For example, an ephemeral message SHOULD not be persisted by the Waku network. If this attribute is set to true, the message SHOULD be interpreted as ephemeral. If, instead, the attribute is omitted or set to false, the message SHOULD be interpreted as non-ephemeral.
Wire Format # The Waku message wire format is specified using protocol buffers v3.
syntax = \u0026#34;proto3\u0026#34;; message WakuMessage { bytes payload = 1; string content_topic = 2; optional uint32 version = 3; optional sint64 timestamp = 10; optional bytes meta = 11; optional bool ephemeral = 31; } An example proto file following this specification can be found here (vacp2p/waku).
Payload encryption # The Waku message payload MAY be encrypted. The message version attribute indicates the schema used to encrypt the payload data.
Version 0: The payload SHOULD be interpreted as unencrypted; additionally, it CAN indicate that the message payload has been encrypted at the application layer.
Version 1: The payload SHOULD be encrypted using Waku v1 payload encryption specified in 26/WAKU-PAYLOAD. This provides asymmetric and symmetric encryption. The key agreement is performed out of band. And provides an encrypted signature and padding for some form of unlinkability.
Version 2: The payload SHOULD be encoded according to 35/WAKU2-NOISE. Waku Noise protocol provides symmetric encryption and asymmetric key exchange.
Any version value not included in this list is reserved for future specification. And, in this case, the payload SHOULD be interpreted as unencrypted by the Waku layer.
Whisper/Waku v1 envelope compatibility # Whisper/Waku v1 envelopes are compatible with Waku v2 messages format.
Whisper/Waku v1 topic field SHOULD be mapped to Waku v2 message\u0026rsquo;s content_topic attribute. Whisper/Waku v1 data field SHOULD be mapped to Waku v2 message\u0026rsquo;s payload attribute. Waku v2 implements a pub/sub messaging pattern over libp2p. This makes redundant some Whisper/Waku v1 envelope fields (e.g., expiry, ttl, topic, etc.), so they can be ignored.
Deterministic message hashing # In Protocol Buffers v3, the deterministic serialization is not canonical across the different implementations and languages. It is also unstable across different builds with schema changes due to unknown fields.
To overcome this interoperability limitation, a Waku v2 message\u0026rsquo;s hash MUST be computed following this schema:
message_hash = sha256(concat(pubsub_topic, message.payload, message.content_topic, message.meta, message.timestamp)) If an optional attribute, such as meta, is absent, the concatenation of attributes SHOULD exclude it. This recommendation is made to ensure that the concatenation process proceeds smoothly when certain attributes are missing and to maintain backward compatibility.
This hashing schema is deemed appropriate for use cases where a cross-implementation deterministic hash is needed, such as message deduplication and integrity validation. The collision probability offered by this hashing schema can be considered negligible. This is due to the deterministic concatenation order of the message attributes, coupled with using a SHA-2 (256-bit) hashing algorithm.
Test vectors # Waku message hash computation (meta size of 12 bytes):
pubsub_topic = \u0026#34;/waku/2/default-waku/proto\u0026#34; (0x2f77616b752f322f64656661756c742d77616b752f70726f746f) message.payload = 0x010203045445535405060708 message.content_topic = \u0026#34;/waku/2/default-content/proto\u0026#34; (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f) message.meta = 0x73757065722d736563726574 message.timestamp = 0x175789bfa23f8400 message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05 Waku message hash computation (meta size of 64 bytes):
pubsub_topic = \u0026#34;/waku/2/default-waku/proto\u0026#34; (0x2f77616b752f322f64656661756c742d77616b752f70726f746f) message.payload = 0x010203045445535405060708 message.content_topic = \u0026#34;/waku/2/default-content/proto\u0026#34; (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f) message.meta = 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f message.timestamp = 0x175789bfa23f8400 message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27 Waku message hash computation (meta attribute not present):
pubsub_topic = \u0026#34;/waku/2/default-waku/proto\u0026#34; (0x2f77616b752f322f64656661756c742d77616b752f70726f746f) message.payload = 0x010203045445535405060708 message.content_topic = \u0026#34;/waku/2/default-content/proto\u0026#34; (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f) message.meta = \u0026lt;not-present\u0026gt; message.timestamp = 0x175789bfa23f8400 message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8 Waku message hash computation (payload length 0):
pubsub_topic = \u0026#34;/waku/2/default-waku/proto\u0026#34; (0x2f77616b752f322f64656661756c742d77616b752f70726f746f) message.payload = [] message.content_topic = \u0026#34;/waku/2/default-content/proto\u0026#34; (0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f) message.meta = 0x73757065722d736563726574 message.timestamp = 0x175789bfa23f8400 message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de4 Security Considerations # Confidentiality, integrity, and authenticity # The level of confidentiality, integrity, and authenticity of the Waku message payload is discretionary. Accordingly, the application layer shall utilize the encryption and signature schemes supported by Waku v2 to meet the application-specific privacy needs.
Reliability of the timestamp attribute # The Waku message timestamp attribute is set by the sender. Therefore, because message timestamps aren’t independently verified, this attribute is prone to exploitation and misuse. It should not solely be relied upon for operations such as message ordering. For example, a malicious actor can arbitrarily set the timestamp of a Waku message to a high value so that it always shows up as the most recent message in a chat application. Applications using Waku messages’ timestamp attribute are recommended to use additional methods for more robust message ordering. An example of how to deal with message ordering against adversarial message timestamps can be found in the Status protocol, see 6/PAYLOADS.
Reliability of the ephemeral attribute # The Waku message ephemeral attribute is set by the sender. Since there is currently no incentive mechanism for network participants to behave correctly, this attribute is inherently insecure. A malicious actor can tamper with the value of a Waku message’s ephemeral attribute, and the receiver would not be able to verify the integrity of the message.
Copyright # Copyright and related rights waived via CC0.
References # 6/WAKU1 Google Protocol buffers v3 26/WAKU-PAYLOAD 35/WAKU2-NOISE 6/PAYLOADS `}),e.add({id:7,href:"/spec/15/",title:"15/WAKU-BRIDGE",section:"Docs",content:`A bridge between Waku v1 and Waku v2.
Bridge # A bridge requires supporting both Waku versions:
Waku v1 - using devp2p RLPx protocol Waku v2 - using libp2p protocols Packets received on the Waku v1 network SHOULD be published just once on the Waku v2 network. More specifically, the bridge SHOULD publish this through the Waku Relay (PubSub domain).
Publishing such packet will require the creation of a new Message with a new WakuMessage as data field. The data and topic field from the Waku v1 Envelope MUST be copied to the payload and contentTopic fields of the WakuMessage. Other fields such as nonce, expiry and ttl will be dropped as they become obsolete in Waku v2.
Before this is done, the usual envelope verification still applies:
Expiry \u0026amp; future time verification PoW verification Size verification Bridging SHOULD occur through the WakuRelay, but it MAY also be done on other Waku v2 protocols (e.g. WakuFilter). The latter is however not advised as it will increase the complexity of the bridge and because of the Security Considerations explained further below.
Packets received on the Waku v2 network SHOULD be posted just once on the Waku v1 network. The Waku v2 WakuMessage contains only the payload and contentTopic fields. The bridge MUST create a new Waku v1 Envelope and copy over the payload and contentFilter fields to the data and topic fields. Next, before posting on the network, the bridge MUST set a new expiry and ttl and do the PoW nonce calculation.
Security Considerations # As mentioned above, a bridge will be posting new Waku v1 envelopes, which requires doing the PoW nonce calculation.
This could be a DoS attack vector, as the PoW calculation will make it more expensive to post the message compared to the original publishing on the Waku v2 network. Low PoW setting will lower this problem, but it is likely that it is still more expensive.
For this reason, bridges SHOULD probably be run independently of other nodes, so that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2 traffic.
Bridging functionality SHOULD also be carefully implemented so that messages do not bounce back and forth between the two networks. The bridge SHOULD properly track messages with a seen filter so that no amplification can be achieved here.
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:8,href:"/spec/16/",title:"16/WAKU2-RPC",section:"Docs",content:` Introduction # This specification describes the JSON-RPC API that Waku v2 nodes MAY adhere to. Refer to the Waku v2 specification for more information on Waku v2.
Wire Protocol # Transport # Nodes SHOULD expose an accessible JSON-RPC API. The JSON-RPC version SHOULD be 2.0. Below is an example request:
{ \u0026#34;jsonrpc\u0026#34;:\u0026#34;2.0\u0026#34;, \u0026#34;method\u0026#34;:\u0026#34;get_waku_v2_debug_info\u0026#34;, \u0026#34;params\u0026#34;:[], \u0026#34;id\u0026#34;:1 } Fields # Field Description jsonrpc Contains the used JSON-RPC version (Default: 2.0) method Contains the JSON-RPC method that is being called params An array of parameters for the request id The request ID Types # In this specification, the primitive types Boolean, String, Number and Null, as well as the structured types Array and Object, are to be interpreted according to the JSON-RPC specification. It also adopts the same capitalisation conventions.
The following structured types are defined for use throughout the document:
WakuMessage # Refer to Waku Message specification for more information.
WakuMessage is an Object containing the following fields:
Field Type Inclusion Description payload String mandatory The message payload as a base64 (with padding) encoded data string contentTopic String optional Message content topic for optional content-based filtering version Number optional Message version. Used to indicate type of payload encryption. Default version is 0 (no payload encryption). timestamp Number optional The time at which the message is generated by its sender. This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. ephemeral Boolean optional This flag indicates the transient nature of the message. Indicates if the message is eligible to be stored by the store protocol, 13/WAKU2-STORE. Method naming # The JSON-RPC methods in this document are designed to be mappable to HTTP REST endpoints. Method names follow the pattern \u0026lt;method_type\u0026gt;_waku_\u0026lt;protocol_version\u0026gt;_\u0026lt;api\u0026gt;_\u0026lt;api_version\u0026gt;_\u0026lt;resource\u0026gt;
\u0026lt;method_type\u0026gt;: prefix of the HTTP method type that most closely matches the JSON-RPC function. Supported method_type values are get, post, put, delete or patch. \u0026lt;protocol_version\u0026gt;: Waku version. Currently v2. \u0026lt;api\u0026gt;: one of the listed APIs below, e.g. store, debug, or relay. \u0026lt;api_version\u0026gt;: API definition version. Currently v1 for all APIs. \u0026lt;resource\u0026gt;: the resource or resource path being addressed The method post_waku_v2_relay_v1_message, for example, would map to the HTTP REST endpoint POST /waku/v2/relay/v1/message.
Debug API # Types # The following structured types are defined for use on the Debug API:
WakuInfo # WakuInfo is an Object containing the following fields:
Field Type Inclusion Description listenAddresses Array[String] mandatory Listening addresses of the node enrUri String optional ENR URI of the node WakuInfo # get_waku_v2_debug_v1_info # The get_waku_v2_debug_v1_info method retrieves information about a Waku v2 node
Parameters # none
Response # WakuInfo - information about a Waku v2 node get_waku_v2_debug_v1_version # The get_waku_v2_debug_v1_version method retrieves the version of a Waku v2 node as a string. The version SHOULD follow semantic versioning. In case the node\u0026rsquo;s current build is based on a git commit between semantic versions, the retrieved version string MAY contain the git commit hash alone or in combination with the latest semantic version.
Parameters # none
Response # string - represents the version of a Waku v2 node Relay API # Refer to the Waku Relay specification for more information on the relaying of messages.
post_waku_v2_relay_v1_message # The post_waku_v2_relay_v1_message method publishes a message to be relayed on a PubSub topic
Parameters # Field Type Inclusion Description topic String mandatory The PubSub topic being published on message WakuMessage mandatory The message being relayed Response # Bool - true on success or an error on failure. post_waku_v2_relay_v1_subscriptions # The post_waku_v2_relay_v1_subscriptions method subscribes a node to an array of PubSub topics.
Parameters # Field Type Inclusion Description topics Array[String] mandatory The PubSub topics being subscribed to Response # Bool - true on success or an error on failure. delete_waku_v2_relay_v1_subscriptions # The delete_waku_v2_relay_v1_subscriptions method unsubscribes a node from an array of PubSub topics.
Parameters # Field Type Inclusion Description topics Array[String] mandatory The PubSub topics being unsubscribed from Response # Bool - true on success or an error on failure. get_waku_v2_relay_v1_messages # The get_waku_v2_relay_v1_messages method returns a list of messages that were received on a subscribed PubSub topic after the last time this method was called. The server MUST respond with an error if no subscription exists for the polled topic. If no message has yet been received on the polled topic, the server SHOULD return an empty list. This method can be used to poll a topic for new messages.
Parameters # Field Type Inclusion Description topic String mandatory The PubSub topic to poll for the latest messages Response # Array[WakuMessage] - the latest messages on the polled topic or an error on failure. Relay Private API # The Private API provides functionality to encrypt/decrypt WakuMessage payloads using either symmetric or asymmetric cryptography. This allows backwards compatibility with Waku v1 nodes. It is the API client\u0026rsquo;s responsibility to keep track of the keys used for encrypted communication. Since keys must be cached by the client and provided to the node to encrypt/decrypt payloads, a Private API SHOULD NOT be exposed on non-local or untrusted nodes.
Types # The following structured types are defined for use on the Private API:
KeyPair # KeyPair is an Object containing the following fields:
Field Type Inclusion Description privateKey String mandatory Private key as hex encoded data string publicKey String mandatory Public key as hex encoded data string get_waku_v2_private_v1_symmetric_key # Generates and returns a symmetric key that can be used for message encryption and decryption.
Parameters # none
Response # String - A new symmetric key as hex encoded data string get_waku_v2_private_v1_asymmetric_keypair # Generates and returns a public/private key pair that can be used for asymmetric message encryption and decryption.
Parameters # none
Response # KeyPair - A new public/private key pair as hex encoded data strings post_waku_v2_private_v1_symmetric_message # The post_waku_v2_private_v1_symmetric_message method publishes a message to be relayed on a PubSub topic.
Before being relayed, the message payload is encrypted using the supplied symmetric key. The client MUST provide a symmetric key.
Parameters # Field Type Inclusion Description topic String mandatory The PubSub topic being published on message WakuMessage mandatory The (unencrypted) message being relayed symkey String mandatory The hex encoded symmetric key to use for payload encryption. This field MUST be included if symmetric key cryptography is selected Response # Bool - true on success or an error on failure. post_waku_v2_private_v1_asymmetric_message # The post_waku_v2_private_v1_asymmetric_message method publishes a message to be relayed on a PubSub topic.
Before being relayed, the message payload is encrypted using the supplied public key. The client MUST provide a public key.
Parameters # Field Type Inclusion Description topic String mandatory The PubSub topic being published on message WakuMessage mandatory The (unencrypted) message being relayed publicKey String mandatory The hex encoded public key to use for payload encryption. This field MUST be included if asymmetric key cryptography is selected Response # Bool - true on success or an error on failure. get_waku_v2_private_v1_symmetric_messages # The get_waku_v2_private_v1_symmetric_messages method decrypts and returns a list of messages that were received on a subscribed PubSub topic after the last time this method was called. The server MUST respond with an error if no subscription exists for the polled topic. If no message has yet been received on the polled topic, the server SHOULD return an empty list. This method can be used to poll a topic for new messages.
Before returning the messages, the server decrypts the message payloads using the supplied symmetric key. The client MUST provide a symmetric key.
Parameters # Field Type Inclusion Description topic String mandatory The PubSub topic to poll for the latest messages symkey String mandatory The hex encoded symmetric key to use for payload decryption. This field MUST be included if symmetric key cryptography is selected Response # Array[WakuMessage] - the latest messages on the polled topic or an error on failure. get_waku_v2_private_v1_asymmetric_messages # The get_waku_v2_private_v1_asymmetric_messages method decrypts and returns a list of messages that were received on a subscribed PubSub topic after the last time this method was called. The server MUST respond with an error if no subscription exists for the polled topic. If no message has yet been received on the polled topic, the server SHOULD return an empty list. This method can be used to poll a topic for new messages.
Before returning the messages, the server decrypts the message payloads using the supplied private key. The client MUST provide a private key.
Parameters # Field Type Inclusion Description topic String mandatory The PubSub topic to poll for the latest messages privateKey String mandatory The hex encoded private key to use for payload decryption. This field MUST be included if asymmetric key cryptography is selected Response # Array[WakuMessage] - the latest messages on the polled topic or an error on failure. Store API # Refer to the Waku Store specification for more information on message history retrieval.
Types # The following structured types are defined for use on the Store API:
StoreResponse # StoreResponse is an Object containing the following fields:
Field Type Inclusion Description messages Array[WakuMessage] mandatory Array of retrieved historical messages pagingOptions PagingOptions conditional Paging information from which to resume further historical queries PagingOptions # PagingOptions is an Object containing the following fields:
Field Type Inclusion Description pageSize Number mandatory Number of messages to retrieve per page cursor Index optional Message Index from which to perform pagination. If not included and forward is set to true, paging will be performed from the beginning of the list. If not included and forward is set to false, paging will be performed from the end of the list. forward Bool mandatory true if paging forward, false if paging backward Index # Index is an Object containing the following fields:
Field Type Inclusion Description digest String mandatory A hash for the message at this Index receivedTime Number mandatory UNIX timestamp in nanoseconds at which the message at this Index was received ContentFilter # ContentFilter is an Object containing the following fields:
Field Type Inclusion Description contentTopic String mandatory The content topic of a WakuMessage get_waku_v2_store_v1_messages # The get_waku_v2_store_v1_messages method retrieves historical messages on specific content topics. This method MAY be called with PagingOptions, to retrieve historical messages on a per-page basis. If the request included PagingOptions, the node MUST return messages on a per-page basis and include PagingOptions in the response. These PagingOptions MUST contain a cursor pointing to the Index from which a new page can be requested.
Parameters # Field Type Inclusion Description pubsubTopic String optional The pubsub topic on which a WakuMessage is published contentFilters Array[ContentFilter] optional Array of content filters to query for historical messages startTime Number optional The inclusive lower bound on the timestamp of queried WakuMessages. This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. endTime Number optional The inclusive upper bound on the timestamp of queried WakuMessages. This field holds the Unix epoch time in nanoseconds as a 64-bits integer value. pagingOptions PagingOptions optional Pagination information Response # StoreResponse - the response to a query for historical messages. Filter API # Refer to the Waku Filter specification for more information on content filtering.
Types # The following structured types are defined for use on the Filter API:
ContentFilter # ContentFilter is an Object containing the following fields:
Field Type Inclusion Description contentTopic String mandatory message content topic post_waku_v2_filter_v1_subscription # The post_waku_v2_filter_v1_subscription method creates a subscription in a light node for messages that matches a content filter and, optionally, a PubSub topic.
Parameters # Field Type Inclusion Description contentFilters Array[ContentFilter] mandatory Array of content filters being subscribed to topic String optional Message topic Response # Bool - true on success or an error on failure. delete_waku_v2_filter_v1_subscription # The delete_waku_v2_filter_v1_subscription method removes subscriptions in a light node matching a content filter and, optionally, a PubSub topic.
Parameters # Field Type Inclusion Description contentFilters Array[ContentFilter] mandatory Array of content filters being unsubscribed from topic String optional Message topic Response # Bool - true on success or an error on failure. get_waku_v2_filter_v1_messages # The get_waku_v2_filter_v1_messages method returns a list of messages that were received on a subscribed content topic after the last time this method was called. The server MUST respond with an error if no subscription exists for the polled content topic. If no message has yet been received on the polled content topic, the server SHOULD respond with an empty list. This method can be used to poll a content topic for new messages.
Parameters # Field Type Inclusion Description contentTopic String mandatory The content topic to poll for the latest messages Response # Array[WakuMessage] - the latest messages on the polled content topic or an error on failure. Admin API # The Admin API provides privileged accesses to the internal operations of a Waku v2 node.
Types # The following structured types are defined for use on the Admin API:
WakuPeer # WakuPeer is an Object containing the following fields:
Field Type Inclusion Description multiaddr String mandatory Multiaddress containing this peer\u0026rsquo;s location and identity protocol String mandatory Protocol that this peer is registered for connected bool mandatory true if peer has active connection for this protocol, false if not get_waku_v2_admin_v1_peers # The get_waku_v2_admin_v1_peers method returns an array of peers registered on this node. Since a Waku v2 node may open either continuous or ad hoc connections, depending on the negotiated protocol, these peers may have different connected states. The same peer MAY appear twice in the returned array, if it is registered for more than one protocol.
Parameters # none
Response # Array[WakuPeer] - Array of peers registered on this node post_waku_v2_admin_v1_peers # The post_waku_v2_admin_v1_peers method connects a node to a list of peers.
Parameters # Field Type Inclusion Description peers Array[String] mandatory Array of peer multiaddrs to connect to. Each multiaddr must contain the location and identity addresses of a peer. Response # Bool - true on success or an error on failure. Example usage # Store API # get_waku_v2_store_v1_messages # This method is part of the store API and the specific resources to retrieve are (historical) messages. The protocol (waku) is on v2, whereas the Store API definition is on v1.
get all the historical messages for content topic \u0026quot;/waku/2/default-content/proto\u0026quot;; no paging required Request # curl -d '{\u0026quot;jsonrpc\u0026quot;:\u0026quot;2.0\u0026quot;,\u0026quot;id\u0026quot;:\u0026quot;id\u0026quot;,\u0026quot;method\u0026quot;:\u0026quot;get_waku_v2_store_v1_messages\u0026quot;, \u0026quot;params\u0026quot;:[\u0026quot;\u0026quot;, [{\u0026quot;contentTopic\u0026quot;:\u0026quot;/waku/2/default-content/proto\u0026quot;}]]}' --header \u0026quot;Content-Type: application/json\u0026quot; http://localhost:8545
{ \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;id\u0026#34;, \u0026#34;method\u0026#34;: \u0026#34;get_waku_v2_store_v1_messages\u0026#34;, \u0026#34;params\u0026#34;: [ \u0026#34;\u0026#34;, [ {\u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;} ] ] } Response # { \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;id\u0026#34;, \u0026#34;result\u0026#34;: { \u0026#34;messages\u0026#34;: [ { \u0026#34;payload\u0026#34;: dGVzdDE, \u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;, \u0026#34;version\u0026#34;: 0 }, { \u0026#34;payload\u0026#34;: dGVzdDI, \u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;, \u0026#34;version\u0026#34;: 0 }, { \u0026#34;payload\u0026#34;: dGVzdDM, \u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;, \u0026#34;version\u0026#34;: 0 } ], \u0026#34;pagingInfo\u0026#34;: null }, \u0026#34;error\u0026#34;: null } get a single page of historical messages for content topic \u0026quot;/waku/2/default-content/proto\u0026quot;; 2 messages per page, backward direction. Since this is the initial query, no cursor is provided, so paging will be performed from the end of the list. Request # curl -d '{\u0026quot;jsonrpc\u0026quot;:\u0026quot;2.0\u0026quot;,\u0026quot;id\u0026quot;:\u0026quot;id\u0026quot;,\u0026quot;method\u0026quot;:\u0026quot;get_waku_v2_store_v1_messages\u0026quot;, \u0026quot;params\u0026quot;:[ \u0026quot;\u0026quot;, [{\u0026quot;contentTopic\u0026quot;:\u0026quot;/waku/2/default-content/proto\u0026quot;}],{\u0026quot;pageSize\u0026quot;:2,\u0026quot;forward\u0026quot;:false}]}' --header \u0026quot;Content-Type: application/json\u0026quot; http://localhost:8545
{ \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;id\u0026#34;, \u0026#34;method\u0026#34;: \u0026#34;get_waku_v2_store_v1_messages\u0026#34;, \u0026#34;params\u0026#34;: [ \u0026#34;\u0026#34;, [ {\u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;} ], { \u0026#34;pageSize\u0026#34;: 2, \u0026#34;forward\u0026#34;: false } ] } Response # { \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;id\u0026#34;, \u0026#34;result\u0026#34;: { \u0026#34;messages\u0026#34;: [ { \u0026#34;payload\u0026#34;: dGVzdDI, \u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;, \u0026#34;version\u0026#34;: 0 }, { \u0026#34;payload\u0026#34;: dGVzdDM, \u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;, \u0026#34;version\u0026#34;: 0 } ], \u0026#34;pagingInfo\u0026#34;: { \u0026#34;pageSize\u0026#34;: 2, \u0026#34;cursor\u0026#34;: { \u0026#34;digest\u0026#34;: \u0026#34;abcdef\u0026#34;, \u0026#34;receivedTime\u0026#34;: 1605887187000000000 }, \u0026#34;forward\u0026#34;: false } }, \u0026#34;error\u0026#34;: null } get the next page of historical messages for content topic \u0026quot;/waku/2/default-content/proto\u0026quot;, using the cursor received above; 2 messages per page, backward direction. Request # curl -d '{\u0026quot;jsonrpc\u0026quot;:\u0026quot;2.0\u0026quot;,\u0026quot;id\u0026quot;:\u0026quot;id\u0026quot;,\u0026quot;method\u0026quot;:\u0026quot;get_waku_v2_store_v1_messages\u0026quot;, \u0026quot;params\u0026quot;:[ \u0026quot;\u0026quot;, [{\u0026quot;contentTopic\u0026quot;:\u0026quot;/waku/2/default-content/proto\u0026quot;}],{\u0026quot;pageSize\u0026quot;:2,\u0026quot;cursor\u0026quot;:{\u0026quot;digest\u0026quot;:\u0026quot;abcdef\u0026quot;,\u0026quot;receivedTime\u0026quot;:1605887187000000000},\u0026quot;forward\u0026quot;:false}]}' --header \u0026quot;Content-Type: application/json\u0026quot; http://localhost:8545
{ \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;id\u0026#34;, \u0026#34;method\u0026#34;: \u0026#34;get_waku_v2_store_v1_messages\u0026#34;, \u0026#34;params\u0026#34;: [ \u0026#34;\u0026#34;, [ {\u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;} ], { \u0026#34;pageSize\u0026#34;: 2, \u0026#34;cursor\u0026#34;: { \u0026#34;digest\u0026#34;: \u0026#34;abcdef\u0026#34;, \u0026#34;receivedTime\u0026#34;: 1605887187000000000 }, \u0026#34;forward\u0026#34;: false } ] } Response # { \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;id\u0026#34;: \u0026#34;id\u0026#34;, \u0026#34;result\u0026#34;: { \u0026#34;messages\u0026#34;: [ { \u0026#34;payload\u0026#34;: dGVzdDE, \u0026#34;contentTopic\u0026#34;: \u0026#34;/waku/2/default-content/proto\u0026#34;, \u0026#34;version\u0026#34;: 0 }, ], \u0026#34;pagingInfo\u0026#34;: { \u0026#34;pageSize\u0026#34;: 2, \u0026#34;cursor\u0026#34;: { \u0026#34;digest\u0026#34;: \u0026#34;123abc\u0026#34;, \u0026#34;receivedTime\u0026#34;: 1605866187000000000 }, \u0026#34;forward\u0026#34;: false } }, \u0026#34;error\u0026#34;: null } References # JSON-RPC specification LibP2P Addressing LibP2P PubSub specification - topic descriptor Waku v2 specification IETF RFC 4648 - The Base16, Base32, and Base64 Data Encodings Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:9,href:"/spec/17/",title:"17/WAKU2-RLN-RELAY",section:"Docs",content:`The 17/WAKU2-RLN-RELAY protocol is an extension of 11/WAKU2-RELAY which additionally provides spam protection using Rate Limiting Nullifiers (RLN).
The security objective is to contain spam activity in a GossipSub network by enforcing a global messaging rate to all the peers. Peers that violate the messaging rate are considered spammers and their message is considered spam. Spammers are also financially punished and removed from the system.
Motivation # In open and anonymous p2p messaging networks, one big problem is spam resistance. Existing solutions, such as Whisper’s proof of work are computationally expensive hence not suitable for resource-limited nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.
We augment the 11/WAKU2-RELAY protocol with a novel construct of RLN to enable an efficient economic spam prevention mechanism that can be run in resource-constrained environments.
Flow # The messaging rate is defined by the period which indicates how many messages can be sent in a given period. We define an epoch as $\\lceil$ unix_time / period $\\rceil$. For example, if unix_time is 1644810116 and we set period to 30, then epoch is $\\lceil$(unix_time/period)$\\rceil$ = 54827003. Note that epoch refers to epoch in RLN and not Unix epoch. This means a message can only be sent every period, where period is up to the application. See see section Recommended System Parameters for some recommended ways to set a sensible period value depending on the application. Peers subscribed to a spam-protected pubsubTopic are only allowed to send one message per epoch. The higher-level layers adopting 17/WAKU2-RLN-RELAY MAY choose to enforce the messaging rate for WakuMessages with a specific contentTopic published on a pubsubTopic.
Setup and Registration # Peers subscribed to a specific pubsubTopic form a RLN group.
Peers MUST be registered to the RLN group to be able to publish messages. Registration is moderated through a smart contract deployed on the Ethereum blockchain. Each peer has an RLN key pair denoted by sk and pk. The secret key sk is secret data and MUST be persisted securely by the peer. The state of the membership contract contains the list of registered members\u0026rsquo; public identity keys i.e., pks. For the registration, a peer creates a transaction that invokes the registration function of the contract via which registers its pk in the group. The transaction also transfers some amount of ether to the contract to be staked. This amount is denoted by staked_fund and is a system parameter. The peer who has the secret key sk associated with a registered pk would be able to withdraw a portion reward_portion of the staked fund by providing valid proof. reward_portion is also a system parameter.
Note that sk is initially only known to its owning peer however, it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch. An overview of registration is illustrated in Figure 1.
Publishing # To publish at a given epoch, the publishing peer proceeds based on the regular 11/WAKU2-RELAY protocol.
However, to protect against spamming, each WakuMessage (which is wrapped inside the data field of a PubSub message) MUST carry a RateLimitProof with the following fields. Section Payload covers the details about the type and encoding of these fields.
The merkle_root contains the root of the Merkle tree.
The epoch represents the current epoch.
The nullifier is an internal nullifier acting as a fingerprint that allows specifying whether two messages are published by the same peer during the same epoch. The nullifier is a deterministic value derived from sk and epoch therefore any two messages issued by the same peer (i.e., using the same sk) for the same epoch are guaranteed to have identical nullifiers.
The share_x and share_y can be seen as partial disclosure of peer\u0026rsquo;s sk for the intended epoch. They are derived deterministically from peer\u0026rsquo;s sk and current epoch using Shamir secret sharing scheme. If a peer discloses more than one such pair (share_x, share_y) for the same epoch, it would allow full disclosure of its sk and hence get access to its staked fund in the membership contract.
The proof field is a zero-knowledge proof signifying that:
The message owner is the current member of the group i.e., her/his identity commitment key pk is part of the membership group Merkle tree with the root merkle_root. share_x and share_y are correctly computed. The nullifier is constructed correctly. For more details about the proof generation check RLN The proof generation relies on the knowledge of two pieces of private information i.e., sk and authPath. The authPath is a subset of Merkle tree nodes by which a peer can prove the inclusion of its pk in the group. The proof generation also requires a set of public inputs which are: the Merkle tree root merkle_root, the current epoch, and the message for which the proof is going to be generated. In 17/WAKU2-RLN-RELAY, the message is the concatenation of WakuMessage\u0026rsquo;s payload filed and its contentTopic i.e., payload||contentTopic. Group Synchronization # Proof generation relies on the knowledge of Merkle tree root merkle_root and authPath which both require access to the membership Merkle tree. Getting access to the Merkle tree can be done in various ways. One way is that all the peers construct the tree locally. This can be done by listening to the registration and deletion events emitted by the membership contract. Peers MUST update the local Merkle tree on a per-block basis. This is discussed further in the Merkle Root Validation section.
Another approach for synchronizing the state of slashed pks is to disseminate such information through a p2p GossipSub network to which all peers are subscribed. This is in addition to sending the deletion transaction to the membership contract. The benefit of an off-chain slashing is that it allows real-time removal of spammers as opposed to on-chain slashing in which peers get informed with a delay, where the delay is due to mining the slashing transaction. For the group synchronization, one important security consideration is that peers MUST make sure they always use the most recent Merkle tree root in their proof generation. The reason is that using an old root can allow inference about the index of the user\u0026rsquo;s pk in the membership tree hence compromising user privacy and breaking message unlinkability.
Routing # Upon the receipt of a PubSub message via 11/WAKU2-RELAY protocol, the routing peer parses the data field as a WakuMessage and gets access to the RateLimitProof field.
The peer then validates the RateLimitProof as explained next.
Epoch Validation # If the epoch attached to the message is more than max_epoch_gap apart from the routing peer\u0026rsquo;s current epoch then the message is discarded and considered invalid. This is to prevent a newly registered peer from spamming the system by messaging for all the past epochs. max_epoch_gap is a system parameter for which we provide some recommendations in section Recommended System Parameters.
Merkle Root Validation # The routing peers MUST check whether the provided Merkle root in the RateLimitProof is valid. It can do so by maintaining a local set of valid Merkle roots, which consist of acceptable_root_window_size past roots. These roots refer to the final state of the Merkle tree after a whole block consisting of group changes is processed. The Merkle roots are updated on a per-block basis instead of a per-event basis. This is done because if Merkle roots are updated on a per-event basis, some peers could send messages with a root that refers to a Merkle tree state that might get invalidated while the message is still propagating in the network, due to many registrations happening during this time frame. By updating roots on a per-block basis instead, we will have only one root update per-block processed, regardless on how many registrations happened in a block, and peers will be able to successfully propagate messages in a time frame corresponding to roughly the size of the roots window times the block mining time.
Atomic processing of the blocks are necessary so that even if the peer is unable to process one event, the previous roots remain valid, and can be used to generate valid RateLimitProof\u0026rsquo;s.
This also allows peers which are not well connected to the network to be able to send messages, accounting for network delay. This network delay is related to the nature of asynchronous network conditions, which means that peers see membership changes asynchronously, and therefore may have differing local Merkle trees. See Recommended System Parameters on choosing an appropriate acceptable_root_window_size.
Proof Verification # The routing peers MUST check whether the zero-knowledge proof proof is valid. It does so by running the zk verification algorithm as explained in RLN. If proof is invalid then the message is discarded.
Spam detection # To enable local spam detection and slashing, routing peers MUST record the nullifier, share_x, and share_y of incoming messages which are not discarded i.e., not found spam or with invalid proof or epoch. To spot spam messages, the peer checks whether a message with an identical nullifier has already been relayed.
If such a message exists and its share_x and share_y components are different from the incoming message, then slashing takes place. That is, the peer uses the share_x and share_y of the new message and the share'_x and share'_y of the old record to reconstruct the sk of the message owner. The sk then can be used to delete the spammer from the group and withdraw a portion reward_portion of its staked fund. If the share_x and share_y fields of the previously relayed message are identical to the incoming message, then the message is a duplicate and shall be discarded. If none is found, then the message gets relayed. An overview of the routing procedure and slashing is provided in Figure 2.
Payloads # Payloads are protobuf messages implemented using protocol buffers v3. Nodes MAY extend the 14/WAKU2-MESSAGE with a rate_limit_proof field to indicate that their message is not spam.
syntax = \u0026#34;proto3\u0026#34;; message RateLimitProof { bytes proof = 1; bytes merkle_root = 2; bytes epoch = 3; bytes share_x = 4; bytes share_y = 5; bytes nullifier = 6; } message WakuMessage { bytes payload = 1; string content_topic = 2; optional uint32 version = 3; optional sint64 timestamp = 10; optional bool ephemeral = 31; + optional bytes rate_limit_proof = 21; } WakuMessage # rate_limit_proof holds the information required to prove that the message owner has not exceeded the message rate limit.
RateLimitProof # Below is the description of the fields of RateLimitProof and their types.
Parameter Type Description proof array of 256 bytes the zkSNARK proof as explained in the Publishing process merkle_root array of 32 bytes in little-endian order the root of membership group Merkle tree at the time of publishing the message share_x and share_y array of 32 bytes each Shamir secret shares of the user\u0026rsquo;s secret identity key sk . share_x is the Poseidon hash of the WakuMessage\u0026rsquo;s payload concatenated with its contentTopic . share_y is calculated using Shamir secret sharing scheme nullifier array of 32 bytes internal nullifier derived from epoch and peer\u0026rsquo;s sk as explained in RLN construct Recommended System Parameters # The system parameters are summarized in the following table, and the recommended values for a subset of them are presented next.
Parameter Description period the length of epoch in seconds staked_fund the amount of wei to be staked by peers at the registration reward_portion the percentage of staked_fund to be rewarded to the slashers max_epoch_gap the maximum allowed gap between the epoch of a routing peer and the incoming message acceptable_root_window_size The maximum number of past Merkle roots to store Epoch Length # A sensible value for the period depends on the application for which the spam protection is going to be used. For example, while the period of 1 second i.e., messaging rate of 1 per second, might be acceptable for a chat application, might be too low for communication among Ethereum network validators. One should look at the desired throughput of the application to decide on a proper period value. In the proof of concept implementation of 17/WAKU2-RLN-RELAY protocol which is available in nim-waku, the period is set to 1 second. Nevertheless, this value is also subject to change depending on user experience.
Maximum Epoch Gap # We discussed in the Routing section that the gap between the epoch observed by the routing peer and the one attached to the incoming message should not exceed a threshold denoted by max_epoch_gap . The value of max_epoch_gap can be measured based on the following factors.
Network transmission delay Network_Delay: the maximum time that it takes for a message to be fully disseminated in the GossipSub network. Clock asynchrony Clock_Asynchrony: The maximum difference between the Unix epoch clocks perceived by network peers which can be due to clock drifts. With a reasonable approximation of the preceding values, one can set max_epoch_gap as max_epoch_gap $= \\lceil \\frac{\\text{Network Delay} + \\text{Clock Asynchrony}}{\\text{Epoch Length}}\\rceil$ where period is the length of the epoch in seconds. Network_Delay and Clock_Asynchrony MUST have the same resolution as period . By this formulation, max_epoch_gap indeed measures the maximum number of epochs that can elapse since a message gets routed from its origin to all the other peers in the network.
acceptable_root_window_size depends upon the underlying chain\u0026rsquo;s average blocktime, block_time
The lower bound for the acceptable_root_window_size SHOULD be set as $acceptable_root_window_size=(Network_Delay)/block_time$
Network_Delay MUST have the same resolution as block_time.
By this formulation, acceptable_root_window_size will provide a lower bound of how many roots can be acceptable by a routing peer.
The acceptable_root_window_size should indicate how many blocks may have been mined during the time it takes for a peer to receive a message. This formula represents a lower bound of the number of acceptable roots.
Copyright # Copyright and related rights waived via CC0.
References # RLN documentation Public inputs to the RLN circuit Shamir secret sharing scheme used in RLN RLN internal nullifier `}),e.add({id:10,href:"/spec/18/",title:"18/WAKU2-SWAP",section:"Docs",content:`This specification outlines how we do accounting and settlement based on the provision and usage of resources, most immediately bandwidth usage and/or storing and retrieving of Waku message. This enables nodes to cooperate and efficiently share resources, and in the case of unequal nodes to settle the difference through a relaxed payment mechanism in the form of sending cheques.
Protocol identifier*: /vac/waku/swap/2.0.0-beta1
Motivation # The Waku network makes up a service network, and some nodes provide a useful service to other nodes. We want to account for that, and when imbalances arise, settle this. The core of this approach has some theoretical backing in game theory, and variants of it have practically been proven to work in systems such as Bittorrent. The specific model use was developed by the Swarm project (previously part of Ethereum), and we re-use contracts that were written for this purpose.
By using a delayed payment mechanism in the form of cheques, a barter-like mechanism can arise, and nodes can decide on their own policy as opposed to be strictly tied to a specific payment scheme. Additionally, this delayed settlement eases requirements on the underlying network in terms of transaction speed or costs.
Theoretically, nodes providing and using resources over a long, indefinite, period of time can be seen as an iterated form of Prisoner\u0026rsquo;s Dilemma (PD). Specifically, and more intuitively, since we have a cost and benefit profile for each provision/usage (of Waku Message\u0026rsquo;s, e.g.), and the pricing can be set such that mutual cooperation is incentivized, this can be analyzed as a form of donations game.
Game Theory - Iterated prisoner\u0026rsquo;s dilemma / donation game # What follows is a sketch of what the game looks like between two nodes. We can look at it as a special case of iterated prisoner\u0026rsquo;s dilemma called a Donation game where each node can cooperate with some benefit b at a personal cost c, where b\u0026gt;c.
From A\u0026rsquo;s point of view:
A/B Cooperate Defect Cooperate b-c -c Defect b 0 What this means is that if A and B cooperates, A gets some benefit b minus a cost c. If A cooperates and B defects she only gets the cost, and if she defects and B cooperates A only gets the benefit. If both defect they get neither benefit nor cost.
The generalized form of PD is:
A/B Cooperate Defect Cooperate R S Defect T P With R=reward, S=Sucker\u0026rsquo;s payoff, T=temptation, P=punishment
And the following holds:
T\u0026gt;R\u0026gt;P\u0026gt;S 2R\u0026gt;T+S In our case, this means b\u0026gt;b-c\u0026gt;0\u0026gt;-c and 2(b-c)\u0026gt; b-c which is trivially true.
As this is an iterated game with no clear finishing point in most circumstances, a tit-for-tat strategy is simple, elegant and functional. To be more theoretically precise, this also requires reasonable assumptions on error rate and discount parameter. This captures notions such as \u0026ldquo;does the perceived action reflect the intended action\u0026rdquo; and \u0026ldquo;how much do you value future (uncertain) actions compared to previous actions\u0026rdquo;. See Axelrod - Evolution of Cooperation (book) for more details. In specific circumstances, nodes can choose slightly different policies if there\u0026rsquo;s a strong need for it. A policy is simply how a node chooses to act given a set of circumstances.
A tit-for-tat strategy basically means:
cooperate first (perform service/beneficial action to other node) defect when node stops cooperating (disconnect and similar actions), i.e. when it stops performing according to set parameters re settlement resume cooperation if other node does so This can be complemented with node selection mechanisms.
SWAP protocol overview # We use SWAP for accounting and settlement in conjunction with other request/reply protocols in Waku v2, where accounting is done in a pairwise manner. It is an acronym with several possible meanings (as defined in the Book of Swarm), for example:
service wanted and provided settle with automated payments send waiver as payment start without a penny This approach is based on communicating payment thresholds and sending cheques as indications of later payments. Communicating payment thresholds MAY be done out-of-band or as part of the handshake. Sending cheques is done once payment threshold is hit.
See Book of Swarm section 3.2. on Peer-to-peer accounting etc., for more context and details.
Accounting # Nodes perform their own accounting for each relevant peer based on some \u0026ldquo;volume\u0026rdquo;/bandwidth metric. For now we take this to mean the number of WakuMessages exchanged.
Additionally, a price is attached to each unit. Currently, this is simply a \u0026ldquo;karma counter\u0026rdquo; and equal to 1 per message.
Each accounting balance SHOULD be w.r.t. to a given protocol it is accounting for.
NOTE: This may later be complemented with other metrics, either as part of SWAP or more likely outside of it. For example, online time can be communicated and attested to as a form of enhanced quality of service to inform peer selection.
Flow # Assuming we have two store nodes, one operating mostly as a client (A) and another as server (B).
Node A performs a handshake with B node. B node responds and both nodes communicate their payment threshold. Node A and B creates an accounting entry for the other peer, keep track of peer and current balance. Node A issues a HistoryRequest, and B responds with a HistoryResponse. Based on the number of WakuMessages in the response, both nodes update their accounting records. When payment threshold is reached, Node A sends over a cheque to reach a neutral balance. Settlement of this is currently out of scope, but would occur through a SWAP contract (to be specified). (mock and hard phase). If disconnect threshold is reached, Node B disconnects Node A (mock and hard phase). Note that not all of these steps are mandatory in initial stages, see below for more details. For example, the payment threshold MAY initially be set out of bounds, and policy is only activated in the mock and hard phase.
Protobufs # We use protobuf to specify the handshake and signature. This current protobuf is a work in progress. This is needed for mock and hard phase.
A handshake gives initial information about payment thresholds and possibly other information. A cheque is best thought of as a promise to pay at a later date.
message Handshake { bytes payment_threshold = 1; } // TODO Signature? // Should probably be over the whole Cheque type message Cheque { bytes beneficiary = 1; // TODO epoch time or block time? uint32 date = 2; // TODO ERC20 extension? // For now karma counter uint32 amount = 3; } Incremental integration and roll-out # To incrementally integrate this into Waku v2, we have divided up the roll-out into three phases:
Soft - accounting only Mock - send mock cheques and take word for it Hard Test - blockchain integration and deployed to public testnet (Goerli, Optimism testnet or similar) Hard Main - deployed to a public mainnet An implementation MAY support any of these phases.
Soft phase # In the soft phase only accounting is performed, without consequence for the peers. No disconnect or sending of cheques is performed at this tage.
SWAP protocol is performed in conjunction with another request-reply protocol to account for its usage. It SHOULD be done for 13/WAKU2-STORE and it MAY be done for other request/reply protocols.
A client SHOULD log accounting state per peer and SHOULD indicate when a peer is out of bounds (either of its thresholds met).
Mock phase # In the mock phase, we send mock cheques and send cheques/disconnect peers as appropriate.
If a node reaches a disconnect threshold, which MUST be outside the payment threshold, it SHOULD disconnect the other peer. If a node is within payment balance, the other node SHOULD stay connected to it. If a node receives a valid Cheque it SHOULD update its internal accounting records. If any node behaves badly, the other node is free to disconnect and pick another node. Peer rating is out of scope of this specification. Hard phase # In the hard phase, in addition to sending cheques and activating policy, this is done with blockchain integration on a public testnet. More details TBD.
This also includes settlements where cheques can be redeemed.
References # Prisoner\u0026rsquo;s Dilemma
Donation game
Axelrod - Evolution of Cooperation (book)
Book of Swarm
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:11,href:"/spec/19/",title:"19/WAKU2-LIGHTPUSH",section:"Docs",content:`Protocol identifier: /vac/waku/lightpush/2.0.0-beta1
Motivation and Goals # Light nodes with short connection windows and limited bandwidth wish to publish messages into the Waku network. Additionally, there is sometimes a need for confirmation that a message has been received \u0026ldquo;by the network\u0026rdquo; (here, at least one node).
19/WAKU2-LIGHTPUSH is a request/response protocol for this.
Payloads # syntax = \u0026#34;proto3\u0026#34;; message PushRequest { string pubsub_topic = 1; WakuMessage message = 2; } message PushResponse { bool is_success = 1; // Error messages, etc string info = 2; } message PushRPC { string request_id = 1; PushRequest request = 2; PushResponse response = 3; } Message Relaying # Nodes that respond to PushRequests MUST either relay the encapsulated message via 11/WAKU2-RELAY protocol on the specified pubsub_topic, or forward the PushRequest via 19/LIGHTPUSH on a 44/WAKU2-DANDELION stem. If they are unable to do so for some reason, they SHOULD return an error code in PushResponse.
Security Considerations # Since this can introduce an amplification factor, it is RECOMMENDED for the node relaying to the rest of the network to take extra precautions. This can be done by rate limiting via 17/WAKU2-RLN-RELAY.
Note that the above is currently not fully implemented.
Copyright # Copyright and related rights waived via CC0.
References # 11/WAKU2-RELAY 44/WAKU2-DANDELION `}),e.add({id:12,href:"/spec/2/",title:"2/MVDS",section:"Docs",content:`In this specification, we describe a minimum viable protocol for data synchronization inspired by the Bramble Synchronization Protocol1. This protocol is designed to ensure reliable messaging between peers across an unreliable peer-to-peer (P2P) network where they may be unreachable or unresponsive.
We present a reference implementation2 including a simulation to demonstrate its performance.
Definitions # Term Description Peer The other nodes that a node is connected to. Record Defines a payload element of either the type OFFER, REQUEST, MESSAGE or ACK Node Some process that is able to store data, do processing and communicate for MVDS. Wire Protocol # Secure Transport # This specification does not define anything related to the transport of packets. It is assumed that this is abstracted in such a way that any secure transport protocol could be easily implemented. Likewise, properties such as confidentiality, integrity, authenticity and forward secrecy are assumed to be provided by a layer below.
Payloads # Payloads are implemented using protocol buffers v3.
syntax = \u0026#34;proto3\u0026#34;; package vac.mvds; message Payload { repeated bytes acks = 5001; repeated bytes offers = 5002; repeated bytes requests = 5003; repeated Message messages = 5004; } message Message { bytes group_id = 6001; int64 timestamp = 6002; bytes body = 6003; } The payload field numbers are kept more \u0026ldquo;unique\u0026rdquo; to ensure no overlap with other protocol buffers.
Each payload contains the following fields:
Acks: This field contains a list (can be empty) of message identifiers informing the recipient that sender holds a specific message. Offers: This field contains a list (can be empty) of message identifiers that the sender would like to give to the recipient. Requests: This field contains a list (can be empty) of message identifiers that the sender would like to receive from the recipient. Messages: This field contains a list of messages (can be empty). Message Identifiers: Each message has a message identifier calculated by hashing the group_id, timestamp and body fields as follows:
HASH(\u0026#34;MESSAGE_ID\u0026#34;, group_id, timestamp, body); Group Identifiers: Each message is assigned into a group using the group_id field, groups are independent synchronization contexts between peers.
The current HASH function used is sha256.
Synchronization # State # We refer to state as set of records for the types OFFER, REQUEST and MESSAGE that every node SHOULD store per peer. state MUST NOT contain ACK records as we do not retransmit those periodically. The following information is stored for records:
Type - Either OFFER, REQUEST or MESSAGE Send Count - The amount of times a record has been sent to a peer. Send Epoch - The next epoch at which a record can be sent to a peer. Flow # A maximum of one payload SHOULD be sent to peers per epoch, this payload contains all ACK, OFFER, REQUEST and MESSAGE records for the specific peer. Payloads are created every epoch, containing reactions to previously received records by peers or new records being sent out by nodes.
Nodes MAY have two modes with which they can send records: BATCH and INTERACTIVE mode. The following rules dictate how nodes construct payloads every epoch for any given peer for both modes.
NOTE: A node may send messages both in interactive and in batch mode.
Interactive Mode # A node initially offers a MESSAGE when attempting to send it to a peer. This means an OFFER is added to the next payload and state for the given peer. When a node receives an OFFER, a REQUEST is added to the next payload and state for the given peer. When a node receives a REQUEST for a previously sent OFFER, the OFFER is removed from the state and the corresponding MESSAGE is added to the next payload and state for the given peer. When a node receives a MESSAGE, the REQUEST is removed from the state and an ACK is added to the next payload for the given peer. When a node receives an ACK, the MESSAGE is removed from the state for the given peer. All records that require retransmission are added to the payload, given Send Epoch has been reached. Batch Mode # When a node sends a MESSAGE, it is added to the next payload and the state for the given peer. When a node receives a MESSAGE, an ACK is added to the next payload for the corresponding peer. When a node receives an ACK, the MESSAGE is removed from the state for the given peer. All records that require retransmission are added to the payload, given Send Epoch has been reached. NOTE: Batch mode is higher bandwidth whereas interactive mode is higher latency.
Retransmission # The record of the type Type SHOULD be retransmitted every time Send Epoch is smaller than or equal to the current epoch.
Send Epoch and Send Count MUST be increased every time a record is retransmitted. Although no function is defined on how to increase Send Epoch, it SHOULD be exponentially increased until reaching an upper bound where it then goes back to a lower epoch in order to prevent a record\u0026rsquo;s Send Epoch\u0026rsquo;s from becoming too large.
NOTE: We do not retransmission ACKs as we do not know when they have arrived, therefore we simply resend them every time we receive a MESSAGE.
Formal Specification # MVDS has been formally specified using TLA+: https://github.com/vacp2p/formalities/tree/master/MVDS.
Acknowledgments # Preston van Loon Greg Markou Rene Nayman Jacek Sieka Copyright # Copyright and related rights waived via CC0.
Footnotes # akwizgran et al. BSP. Briar.\u0026#160;\u0026#x21a9;\u0026#xfe0e;
https://github.com/vacp2p/mvds\u0026#160;\u0026#x21a9;\u0026#xfe0e;
`}),e.add({id:13,href:"/spec/20/",title:"20/TOY-ETH-PM",section:"Docs",content:`Content Topics:
Public Key Broadcast: /eth-pm/1/public-key/proto, Private Message: /eth-pm/1/private-message/proto. This specification explains the Toy Ethereum Private Message protocol which enables a peer to send an encrypted message to another peer using the Waku v2 network, and the peer\u0026rsquo;s Ethereum address.
The main purpose of this specification is to demonstrate how Waku v2 can be used for encrypted messaging purposes, using Ethereum accounts for identity. This protocol caters for Web3 wallets restrictions, allowing it to be implemented only using standard Web3 API. In the current state, the protocol has privacy and features limitations, has not been audited and hence is not fit for production usage. We hope this can be an inspiration for developers wishing to build on top of Waku v2.
Goal # Alice wants to send an encrypted message to Bob, where only Bob can decrypt the message. Alice only knows Bob\u0026rsquo;s Ethereum Address.
Variables # Here are the variables used in the protocol and their definition:
B is Bob\u0026rsquo;s Ethereum address (or account), b is the private key of B, and is only known by Bob. B' is Bob\u0026rsquo;s Encryption Public Key, for which b' is the private key. M is the private message that Alice sends to Bob. Design Requirements # The proposed protocol MUST adhere to the following design requirements:
Alice knows Bob\u0026rsquo;s Ethereum address, Bob is willing to participate to Eth-PM, and publishes B', Bob\u0026rsquo;s ownership of B' MUST be verifiable, Alice wants to send message M to Bob, Bob SHOULD be able to get M using 10/WAKU2, Participants only have access to their Ethereum Wallet via the Web3 API, Carole MUST NOT be able to read M\u0026rsquo;s content even if she is storing it or relaying it, Waku Message Version 1 Asymmetric Encryption is used for encryption purposes. Limitations # Alice\u0026rsquo;s details are not included in the message\u0026rsquo;s structure, meaning that there is no programmatic way for Bob to reply to Alice or verify her identity.
Private messages are sent on the same content topic for all users. As the recipient data is encrypted, all participants must decrypt all messages which can lead to scalability issues.
This protocol does not guarantee Perfect Forward Secrecy nor Future Secrecy: If Bob\u0026rsquo;s private key is compromised, past and future messages could be decrypted. A solution combining regular X3DH bundle broadcast with Double Ratchet encryption would remove these limitations; See the Status secure transport spec for an example of a protocol that achieves this in a peer-to-peer setting.
Bob MUST decide to participate in the protocol before Alice can send him a message. This is discussed in more in details in Consideration for a non-interactive/uncoordinated protocol
The protocol # Generate Encryption KeyPair # First, Bob needs to generate a keypair for Encryption purposes.
Bob SHOULD get 32 bytes from a secure random source as Encryption Private Key, b'. Then Bob can compute the corresponding SECP-256k1 Public Key, B'.
Broadcast Encryption Public Key # For Alice to encrypt messages for Bob, Bob SHOULD broadcast his Encryption Public Key B'. To prove that the Encryption Public Key B' is indeed owned by the owner of Bob\u0026rsquo;s Ethereum Account B, Bob MUST sign B' using B.
Sign Encryption Public Key # To prove ownership of the Encryption Public Key, Bob must sign it using EIP-712 v3, meaning calling eth_signTypedData_v3 on his Wallet\u0026rsquo;s API.
Note: While v4 also exists, it is not available on all wallets and the features brought by v4 is not needed for the current use case.
The TypedData to be passed to eth_signTypedData_v3 MUST be as follows, where:
encryptionPublicKey is Bob\u0026rsquo;s Encryption Public Key, B', in hex format, without 0x prefix. bobAddress is Bob\u0026rsquo;s Ethereum address, corresponding to B, in hex format, with 0x prefix. const typedData = { domain: { chainId: 1, name: \u0026#39;Ethereum Private Message over Waku\u0026#39;, version: \u0026#39;1\u0026#39;, }, message: { encryptionPublicKey: encryptionPublicKey, ownerAddress: bobAddress, }, primaryType: \u0026#39;PublishEncryptionPublicKey\u0026#39;, types: { EIP712Domain: [ { name: \u0026#39;name\u0026#39;, type: \u0026#39;string\u0026#39; }, { name: \u0026#39;version\u0026#39;, type: \u0026#39;string\u0026#39; }, { name: \u0026#39;chainId\u0026#39;, type: \u0026#39;uint256\u0026#39; }, ], PublishEncryptionPublicKey: [ { name: \u0026#39;encryptionPublicKey\u0026#39;, type: \u0026#39;string\u0026#39; }, { name: \u0026#39;ownerAddress\u0026#39;, type: \u0026#39;string\u0026#39; }, ], }, } Public Key Message # The resulting signature is then included in a PublicKeyMessage, where
encryption_public_key is Bob\u0026rsquo;s Encryption Public Key B', not compressed, eth_address is Bob\u0026rsquo;s Ethereum Address B, signature is the EIP-712 as described above. syntax = \u0026#34;proto3\u0026#34;; message PublicKeyMessage { bytes encryption_public_key = 1; bytes eth_address = 2; bytes signature = 3; } This MUST be wrapped in a Waku Message version 0, with the Public Key Broadcast content topic. Finally, Bob SHOULD publish the message on Waku v2.
Consideration for a non-interactive/uncoordinated protocol # Alice has to get Bob\u0026rsquo;s public Key to send a message to Bob. Because an Ethereum Address is part of the hash of the public key\u0026rsquo;s account, it is not enough in itself to deduce Bob\u0026rsquo;s Public Key.
This is why the protocol dictates that Bob MUST send his Public Key first, and Alice MUST receive it before she can send him a message.
Moreover, nim-waku, the reference implementation of 13/WAKU2-STORE, stores messages for a maximum period of 30 days. This means that Bob would need to broadcast his public key at least every 30 days to be reachable.
Below we are reviewing possible solutions to mitigate this \u0026ldquo;sign up\u0026rdquo; step.
Retrieve the public key from the blockchain # If Bob has signed at least one transaction with his account then his Public Key can be extracted from the transaction\u0026rsquo;s ECDSA signature. The challenge with this method is that standard Web3 Wallet API does not allow Alice to specifically retrieve all/any transaction sent by Bob.
Alice would instead need to use the eth.getBlock API to retrieve Ethereum blocks one by one. For each block, she would need to check the from value of each transaction until she finds a transaction sent by Bob.
This process is resource intensive and can be slow when using services such as Infura due to rate limits in place, which makes it inappropriate for a browser or mobile phone environment.
An alternative would be to either run a backend that can connect directly to an Ethereum node, use a centralized blockchain explorer or use a decentralized indexing service such as The Graph.
Note that these would resolve a UX issue only if a sender wants to proceed with air drops.
Indeed, if Bob does not publish his Public Key in the first place then it can be an indication that he simply does not participate in this protocol and hence will not receive messages.
However, these solutions would be helpful if the sender wants to proceed with an air drop of messages: Send messages over Waku for users to retrieve later, once they decide to participate in this protocol. Bob may not want to participate first but may decide to participate at a later stage and would like to access previous messages. This could make sense in an NFT offer scenario: Users send offers to any NFT owner, NFT owner may decide at some point to participate in the protocol and retrieve previous offers.
Publishing the public in long term storage # Another improvement would be for Bob not having to re-publish his public key every 30 days or less. Similarly to above, if Bob stops publishing his public key then it may be an indication that he does not participate in the protocol anymore.
In any case, the protocol could be modified to store the Public Key in a more permanent storage, such as a dedicated smart contract on the blockchain.
Send Private Message # Alice MAY monitor the Waku v2 to collect Ethereum Address and Encryption Public Key tuples. Alice SHOULD verify that the signatures of PublicKeyMessages she receives are valid as per EIP-712. She SHOULD drop any message without a signature or with an invalid signature.
Using Bob\u0026rsquo;s Encryption Public Key, retrieved via 10/WAKU2, Alice MAY now send an encrypted message to Bob.
If she wishes to do so, Alice MUST encrypt her message M using Bob\u0026rsquo;s Encryption Public Key B', as per 26/WAKU-PAYLOAD Asymmetric Encryption specs.
Alice SHOULD now publish this message on the Private Message content topic.
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:14,href:"/spec/21/",title:"21/WAKU2-FAULT-TOLERANT-STORE",section:"Docs",content:`The reliability of 13/WAKU2-STORE protocol heavily relies on the fact that full nodes i.e., those who persist messages have high availability and uptime and do not miss any messages. If a node goes offline, then it will risk missing all the messages transmitted in the network during that time. In this specification, we provide a method that makes the store protocol resilient in presence of faulty nodes. Relying on this method, nodes that have been offline for a time window will be able to fix the gap in their message history when getting back online. Moreover, nodes with lower availability and uptime can leverage this method to reliably provide the store protocol services as a full node.
Method description # As the first step towards making the 13/WAKU2-STORE protocol fault-tolerant, we introduce a new type of time-based query through which nodes fetch message history from each other based on their desired time window. This method operates based on the assumption that the querying node knows some other nodes in the store protocol which have been online for that targeted time window.
Security Consideration # The main security consideration to take into account while using this method is that a querying node has to reveal its offline time to the queried node. This will gradually result in the extraction of the node\u0026rsquo;s activity pattern which can lead to inference attacks.
Wire Specification # We extend the HistoryQuery protobuf message with two fields of start_time and end_time to signify the time range to be queried.
Payloads # syntax = \u0026#34;proto3\u0026#34;; message HistoryQuery { // the first field is reserved for future use string pubsubtopic = 2; repeated ContentFilter contentFilters = 3; PagingInfo pagingInfo = 4; + sint64 start_time = 5; + sint64 end_time = 6; } HistoryQuery # RPC call to query historical messages.
start_time: this field MAY be filled out to signify the starting point of the queried time window. This field holds the Unix epoch time in nanoseconds.
The messages field of the corresponding HistoryResponse MUST contain historical waku messages whose timestamp is larger than or equal to the start_time.
end_time this field MAY be filled out to signify the ending point of the queried time window. This field holds the Unix epoch time in nanoseconds. The messages field of the corresponding HistoryResponse MUST contain historical waku messages whose timestamp is less than or equal to the end_time.
A time-based query is considered valid if its end_time is larger than or equal to the start_time. Queries that do not adhere to this condition will not get through e.g. an open-end time query in which the start_time is given but no end_time is supplied is not valid. If both start_time and end_time are omitted then no time-window filter takes place.
In order to account for nodes asynchrony, and assuming that nodes may be out of sync for at most 20 seconds (i.e., 20000000000 nanoseconds), the querying nodes SHOULD add an offset of 20 seconds to their offline time window. That is if the original window is [l,r] then the history query SHOULD be made for [start_time: l - 20s, end_time: r + 20s].
Note that HistoryQuery preserves AND operation among the queried attributes. As such, The messages field of the corresponding HistoryResponse MUST contain historical waku messages that satisfy the indicated pubsubtopic AND contentFilters AND the time range [start_time, end_time].
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:15,href:"/spec/22/",title:"22/TOY-CHAT",section:"Docs",content:`Content Topic: /toy-chat/2/huilong/proto.
This specification explains a toy chat example using Waku v2. This protocol is mainly used to:
Dogfood Waku v2, Show an example of how to use Waku v2. Currently, all main Waku v2 implementations support the toy chat protocol: nim-waku, js-waku (NodeJS and web) and go-waku.
Note that this is completely separate from the protocol the Status app is using for its chat functionality.
Design # The chat protocol enables sending and receiving messages in a chat room. There is currently only one chat room, which is tied to the content topic. The messages SHOULD NOT be encrypted.
The contentTopic MUST be set to /toy-chat/2/huilong/proto.
Payloads # syntax = \u0026#34;proto3\u0026#34;; message Chat2Message { uint64 timestamp = 1; string nick = 2; bytes payload = 3; } timestamp: The time at which the message was sent, in Unix Epoch seconds, nick: The nickname of the user sending the message, payload: The text of the messages, UTF-8 encoded. Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:16,href:"/spec/23/",title:"23/WAKU2-TOPICS",section:"Docs",content:`This document outlines recommended usage of topic names in Waku v2. In 10/WAKU2 spec there are two types of topics:
pubsub topics, used for routing Content topics, used for content-based filtering Pubsub Topics # Pubsub topics are used for routing of messages (see 11/WAKU2-RELAY), and can be named implicitly by Waku sharding (see 51/WAKU2-RELAY-SHARDING). This document comprises recommendations for explicitly naming pubsub topics (e.g. when choosing named sharding as specified in 51/WAKU2-RELAY-SHARDING).
Pubsub Topic Format # Pubsub topics SHOULD follow the following structure:
/waku/2/{topic-name}
This namespaced structure makes compatibility, discoverability, and automatic handling of new topics easier.
The first two parts indicate
it relates to the Waku protocol domain, and the version is 2. If applicable, it is RECOMMENDED to structure {topic-name} in a hierarchical way as well.
Note: In previous versions of this document, the structure was /waku/2/{topic-name}/{encoding}. The now deprecated /{encoding} was always set to /proto, which indicated that the data field in pubsub is serialized/encoded as protobuf. The inspiration for this format was taken from Ethereum 2 P2P spec. However, because the payload of messages transmitted over 11/WAKU2-RELAY must be a 14/WAKU2-MESSAGE, which specifies the wire format as protobuf,/proto is the only valid encoding. This makes the /proto indication obsolete. The encoding of the payload field of a Waku Message is indicated by the /{encoding} part of the content topic name. Specifying an encoding is only significant for the actual payload/data field. Waku preserves this option by allowing to specify an encoding for the WakuMessage payload field as part of the content topic name.
Default PubSub Topic # The Waku v2 default pubsub topic is:
/waku/2/default-waku/proto
The {topic name} part is default-waku/proto, which indicates it is default topic for exchanging WakuMessages; /proto remains for backwards compatibility.
Application Specific Names # Larger apps can segregate their pubsub meshes using topics named like:
/waku/2/status/ /waku/2/walletconnect/ This indicates that these networks carry WakuMessages, but for different domains completely.
Named Topic Sharding Example # The following is an example of named sharding, as specified in 51/WAKU2-RELAY-SHARDING.
waku/2/waku-9_shard-0/ ... waku/2/waku-9_shard-9/ This indicates explicitly that the network traffic has been partitioned into 10 buckets.
Content Topics # The other type of topic that exists in Waku v2 is a content topic. This is used for content based filtering. See 14/WAKU2-MESSAGE spec for where this is specified. Note that this doesn\u0026rsquo;t impact routing of messages between relaying nodes, but it does impact how request/reply protocols such as 12/WAKU2-FILTER and 13/WAKU2-STORE are used.
This is especially useful for nodes that have limited bandwidth, and only want to pull down messages that match this given content topic.
Since all messages are relayed using the relay protocol regardless of content topic, you MAY use any content topic you wish without impacting how messages are relayed.
Content Topic Format # The format for content topics is as follows:
/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}
The name of a content topic is application-specific. As an example, here\u0026rsquo;s the content topic used for an upcoming testnet:
/toychat/2/huilong/proto
Content Topic Naming Recommendations # Application names should be unique to avoid conflicting issues with other protocols. Applications should specify their version (if applicable) in the version field. The {content-topic-name} portion of the content topic is up to the application, and depends on the problem domain. It can be hierarchical, for instance to separate content, or to indicate different bandwidth and privacy guarantees. The encoding field indicates the serialization/encoding scheme for the WakuMessage payload field.
Differences with Waku v1 # In 6/WAKU1 there is no actual routing. All messages are sent to all other nodes. This means that we are implicitly using the same pubsub topic that would be something like:
/waku/1/default-waku/rlp Topics in Waku v1 correspond to Content Topics in Waku v2.
Bridging Waku v1 and Waku v2 # To bridge Waku v1 and Waku v2 we have a 15/WAKU-BRIDGE. For mapping Waku v1 topics to Waku v2 content topics, the following structure for the content topic SHOULD be used:
/waku/1/\u0026lt;4bytes-waku-v1-topic\u0026gt;/rfc26 The \u0026lt;4bytes-waku-v1-topic\u0026gt; SHOULD be the lowercase hex representation of the 4-byte Waku v1 topic. A 0x prefix SHOULD be used. /rfc26 indicates that the bridged content is encoded according to RFC 26/WAKU-PAYLOAD. See 15/WAKU-BRIDGE for a description of the bridged fields.
This creates a direct mapping between the two protocols. For example:
/waku/1/0x007f80ff/rfc26 Copyright # Copyright and related rights waived via CC0.
References # 10/WAKU2 spec 11/WAKU2-RELAY 51/WAKU2-RELAY-SHARDING Ethereum 2 P2P spec 14/WAKU2-MESSAGE spec 12/WAKU2-FILTER 13/WAKU2-STORE 6/WAKU1 15/WAKU-BRIDGE 26/WAKU-PAYLOAD `}),e.add({id:17,href:"/spec/24/",title:"24/STATUS-CURATION",section:"Docs",content:` Abstract # This specification is a voting protocol for peers to submit votes to a smart contract. Voting is immutable, this will help avoid sabotage from malicious peers.
Motivation # In open p2p protocol there is an issue with voting off-chain as there is much room for malicious peers to only include votes that support their case when submitting votes to chain.
Proposed solution is to aggregate votes over waku and allow users to submit votes to smart contract that aren\u0026rsquo;t already submitted.
Smart contract # Voting should be finalized on chain so that the finished vote is immutable. Because of that, smart contract needs to be deployed. When votes are submitted smart contract has to verify what votes are properly signed and that sender has correct amount of SNT. When Vote is verified the amount of SNT voted on specific topic by specific sender is saved on chain.
Double voting # Smart contract should also keep a list of all signatures so that no one can send the same vote twice. Another possibility is to allow each sender to only vote once.
Initializing Vote # When someone wants to initialize vote he has to send a transaction to smart contract that will create a new voting session. When initializing a user has to specify type of vote (Addition, Deletion), amount of his initial SNT to submit and public key of community under vote. Smart contract will return a ID which is identifier of voting session. Also there will be function on Smart Contract that when given community public key it will return voting session ID or undefined if community isn\u0026rsquo;t under vote.
Voting # Sending votes # Sending votes is simple every peer is able to send a message to Waku topic specific to given application:
/status-community-directory-curation-vote/1/{voting-session-id}/json vote object that is sent over waku should contain information about:
type Vote = { sender: string // address of the sender vote: string // vote sent eg. \u0026#39;yes\u0026#39; \u0026#39;no\u0026#39; sntAmount: BigNumber //number of snt cast on vote sign: string // cryptographic signature of a transaction (signed fields: sender,vote,sntAmount,nonce,sessionID) nonce: number // number of votes cast from this address on current vote (only if we allow multiple votes from the same sender) sessionID: number // ID of voting session } Aggregating votes # Every peer that is opening specific voting session will listen to votes sent over p2p network, and aggregate them for a single transaction to chain.
Submitting to chain # Every peer that has aggregated at least one vote will be able to send them to smart contract. When someone votes he will aggregate his own vote and will be able to immediately send it.
Peer doesn\u0026rsquo;t need to vote to be able to submit the votes to the chain.
Smart contract needs to verify that all votes are valid (eg. all senders had enough SNT, all votes are correctly signed) and that votes aren\u0026rsquo;t duplicated on smart contract.
Finalizing # Once the vote deadline has expired, the smart contract will not accept votes anymore. Also directory will be updated according to vote results (community added to directory, removed etc.)
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:18,href:"/spec/25/",title:"25/LIBP2P-DNS-DISCOVERY",section:"Docs",content:`25/LIBP2P-DNS-DISCOVERY specifies a scheme to implement libp2p peer discovery via DNS for Waku v2. The generalised purpose is to retrieve an arbitrarily long, authenticated, updateable list of libp2p peers to bootstrap connection to a libp2p network. Since 10/WAKU2 currently specifies use of libp2p peer identities, this method is suitable for a new Waku v2 node to discover other Waku v2 nodes to connect to.
This specification is largely based on EIP-1459, with the only deviation being the type of address being encoded (multiaddr vs enr). Also see this earlier explainer for more background on the suitability of DNS based discovery for Waku v2.
List encoding # The peer list MUST be encoded as a Merkle tree. EIP-1459 specifies the URL scheme to refer to such a DNS node list. This specification uses the same approach, but with a matree scheme:
matree://\u0026lt;key\u0026gt;@\u0026lt;fqdn\u0026gt; where
matree is the selected multiaddr Merkle tree scheme \u0026lt;fqdn\u0026gt; is the fully qualified domain name on which the list can be found \u0026lt;key\u0026gt; is the base32 encoding of the compressed 32-byte binary public key that signed the list. The example URL from EIP-1459, adapted to the above scheme becomes:
matree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@peers.example.org Each entry within the Merkle tree MUST be contained within a DNS TXT record and stored in a subdomain (except for the base URL matree entry). The content of any TXT record MUST be small enough to fit into the 512-byte limit imposed on UDP DNS packets, which limits the number of hashes that can be contained within a branch entry. The subdomain name for each entry is the base32 encoding of the abbreviated keccak256 hash of its text content. See this example of a fully populated tree for more information.
Entry types # The following entry types are derived from EIP-1459 and adapted for use with multiaddrs:
Root entry # The tree root entry MUST use the following format:
matree-root:v1 m=\u0026lt;ma-root\u0026gt; l=\u0026lt;link-root\u0026gt; seq=\u0026lt;sequence number\u0026gt; sig=\u0026lt;signature\u0026gt; where
ma-root and link-root refer to the root hashes of subtrees containing multiaddrs and links to other subtrees, respectively sequence-number is the tree\u0026rsquo;s update sequence number. This number SHOULD increase with each update to the tree. signature is a 65-byte secp256k1 EC signature over the keccak256 hash of the root record content, excluding the sig= part, encoded as URL-safe base64 Branch entry # Branch entries MUST take the format:
matree-branch:\u0026lt;h₁\u0026gt;,\u0026lt;h₂\u0026gt;,...,\u0026lt;hₙ\u0026gt; where
\u0026lt;h₁\u0026gt;,\u0026lt;h₂\u0026gt;,...,\u0026lt;hₙ\u0026gt; are the hashes of other subtree entries Leaf entries # There are two types of leaf entries:
Link entries # For the subtree pointed to by link-root, leaf entries MUST take the format:
matree://\u0026lt;key\u0026gt;@\u0026lt;fqdn\u0026gt; which links to a different list located in another domain.
multiaddr entries # For the subtree pointed to by ma-root, leaf entries MUST take the format:
ma:\u0026lt;multiaddr\u0026gt; which contains the multiaddr of a libp2p peer.
Client protocol # A client MUST adhere to the client protocol as specified in EIP-1459, and adapted for usage with multiaddr entry types below:
To find nodes at a given DNS name a client MUST perform the following steps:
Resolve the TXT record of the DNS name and check whether it contains a valid matree-root:v1 entry. Verify the signature on the root against the known public key and check whether the sequence number is larger than or equal to any previous number seen for that name. Resolve the TXT record of a hash subdomain indicated in the record and verify that the content matches the hash. If the resolved entry is of type: matree-branch: parse the list of hashes and continue resolving them (step 3). ma: import the multiaddr and add it to a local list of discovered nodes. Copyright # Copyright and related rights waived via CC0.
References # 10/WAKU2 EIP-1459: Client Protocol EIP-1459: Node Discovery via DNS libp2p libp2p peer identity Merkle trees `}),e.add({id:19,href:"/spec/26/",title:"26/WAKU2-PAYLOAD",section:"Docs",content:`This specification describes how Waku provides confidentiality, authenticity, and integrity, as well as some form of unlinkability. Specifically, it describes how encryption, decryption and signing works in 6/WAKU1 and in 10/WAKU2 with 14/WAKU-MESSAGE version 1.
This specification effectively replaces 7/WAKU-DATA as well as 6/WAKU1 Payload encryption but written in a way that is agnostic and self-contained for Waku v1 and Waku v2.
Large sections of the specification originate from EIP-627: Whisper spec as well from RLPx Transport Protocol spec (ECIES encryption) with some modifications.
Design requirements # Confidentiality: The adversary should not be able to learn what data is being sent from one Waku node to one or several other Waku nodes. Authenticity: The adversary should not be able to cause Waku endpoint to accept data from any third party as though it came from the other endpoint. Integrity: The adversary should not be able to cause a Waku endpoint to accept data that has been tampered with. Notable, forward secrecy is not provided for at this layer. If this property is desired, a more fully featured secure communication protocol can be used on top, such as Status 5/SECURE-TRANSPORT.
It also provides some form of unlinkability since:
only participants who are able to decrypt a message can see its signature payload are padded to a fixed length Cryptographic primitives # AES-256-GCM (for symmetric encryption) ECIES ECDSA KECCAK-256 ECIES is using the following cryptosystem:
Curve: secp256k1 KDF: NIST SP 800-56 Concatenation Key Derivation Function, with SHA-256 option MAC: HMAC with SHA-256 AES: AES-128-CTR Specification # For 6/WAKU1, the data field is used in the waku envelope, and the field MAY contain the encrypted payload.
For 10/WAKU2, the payload field is used in WakuMessage and MAY contain the encrypted payload.
The fields that are concatenated and encrypted as part of the data (Waku v1) / payload (Waku v2) field are:
flags payload-length payload padding signature ABNF # Using Augmented Backus-Naur form (ABNF) we have the following format:
; 1 byte; first two bits contain the size of payload-length field, ; third bit indicates whether the signature is present. flags = 1OCTET ; contains the size of payload. payload-length = 4*OCTET ; byte array of arbitrary size (may be zero). payload = *OCTET ; byte array of arbitrary size (may be zero). padding = *OCTET ; 65 bytes, if present. signature = 65OCTET data = flags payload-length payload padding [signature] ; This field is called payload in Waku v2 payload = data Signature # Those unable to decrypt the payload/data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialized as the concatenation of the r, s and v parameters of the SECP-256k1 ECDSA signature, in that order. r and s MUST be big-endian encoded, fixed-width 256-bit unsigned. v MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
See Ethereum \u0026ldquo;Yellow paper\u0026rdquo;: Appendix F Signing transactions for more information on signature generation, parameters and public key recovery.
Encryption # Symmetric # Symmetric encryption uses AES-256-GCM for authenticated encryption. The output of encryption is of the form (ciphertext, tag, iv) where ciphertext is the encrypted message, tag is a 16 byte message authentication tag and iv is a 12 byte initialization vector (nonce). The message authentication tag and initialization vector iv field MUST be appended to the resulting ciphertext, in that order. Note that previous specifications and some implementations might refer to iv as nonce or salt.
Asymmetric # Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme (ECIES) with SECP-256k1 public key.
ECIES # This section originates from the RLPx Transport Protocol spec spec with minor modifications.
The cryptosystem used is:
The elliptic curve secp256k1 with generator G. KDF(k, len): the NIST SP 800-56 Concatenation Key Derivation Function. MAC(k, m): HMAC using the SHA-256 hash function. AES(k, iv, m): the AES-128 encryption function in CTR mode. Special notation used: X || Y denotes concatenation of X and Y.
Alice wants to send an encrypted message that can be decrypted by Bob\u0026rsquo;s static private key kB. Alice knows about Bobs static public key KB.
To encrypt the message m, Alice generates a random number r and corresponding elliptic curve public key R = r * G and computes the shared secret S = Px where (Px, Py) = r * KB. She derives key material for encryption and authentication as kE || kM = KDF(S, 32) as well as a random initialization vector iv. Alice sends the encrypted message R || iv || c || d where c = AES(kE, iv , m) and d = MAC(sha256(kM), iv || c) to Bob.
For Bob to decrypt the message R || iv || c || d, he derives the shared secret S = Px where (Px, Py) = kB * R as well as the encryption and authentication keys kE || kM = KDF(S, 32). Bob verifies the authenticity of the message by checking whether d == MAC(sha256(kM), iv || c) then obtains the plaintext as m = AES(kE, iv || c).
Padding # The padding field is used to align data size, since data size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the IV and tag) before encryption (i.e. plain text) SHOULD be a multiple of 256 bytes.
Decoding a message # In order to decode a message, a node SHOULD try to apply both symmetric and asymmetric decryption operations. This is because the type of encryption is not included in the message.
References # 6/WAKU1 10/WAKU2 14/WAKU-MESSAGE version 1 7/WAKU-DATA 6/WAKU1 Payload encryption EIP-627: Whisper spec RLPx Transport Protocol spec (ECIES encryption) Status 5/SECURE-TRANSPORT Augmented Backus-Naur form (ABNF) Ethereum \u0026ldquo;Yellow paper\u0026rdquo;: Appendix F Signing transactions Authenticated encryption Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:20,href:"/spec/27/",title:"27/WAKU2-PEERS",section:"Docs",content:`27/WAKU2-PEERS describes a recommended minimal set of peer storage and peer management features to be implemented by Waku v2 clients.
In this context, peer storage refers to a client\u0026rsquo;s ability to keep track of discovered or statically-configured peers and their metadata. It also deals with matters of peer persistence, or the ability to store peer data on disk to resume state after a client restart.
Peer management is a closely related concept and refers to the set of actions a client MAY choose to perform based on its knowledge of its connected peers, e.g. triggering reconnects/disconnects, keeping certain connections alive, etc.
Peer store # The peer store SHOULD be an in-memory data structure where information about discovered or configured peers are stored. It SHOULD be considered the main source of truth for peer-related information in a Waku v2 client. Clients MAY choose to persist this store on-disk.
Tracked peer metadata # It is RECOMMENDED that a Waku v2 client tracks at least the following information about each of its peers in a peer store:
Metadata Description Public key The public key for this peer. This is related to the libp2p Peer ID. Addresses Known transport layer multiaddrs for this peer. Protocols The libp2p protocol IDs supported by this peer. This can be used to track the client\u0026rsquo;s connectivity to peers supporting different Waku v2 protocols, e.g. 11/WAKU2-RELAY or 13/WAKU2-STORE. Connectivity Tracks the peer\u0026rsquo;s current connectedness state. See Peer connectivity below. Disconnect time The timestamp at which this peer last disconnected. This becomes important when managing peer reconnections Peer connectivity # A Waku v2 client SHOULD track at least the following connectivity states for each of its peers:
NotConnected: The peer has been discovered or configured on this client, but no attempt has yet been made to connect to this peer. This is the default state for a new peer. CannotConnect: The client attempted to connect to this peer, but failed. CanConnect: The client was recently connected to this peer and disconnected gracefully. Connected: The client is actively connected to this peer. This list does not preclude clients from tracking more advanced connectivity metadata, such as a peer\u0026rsquo;s blacklist status (see (18/WAKU2-SWAP)[https://rfc.vac.dev/spec/18/]).
Persistence # A Waku v2 client MAY choose to persist peers across restarts, using any offline storage technology, such as an on-disk database. Peer persistence MAY be used to resume peer connections after a client restart.
Peer management # Waku v2 clients will have different requirements when it comes to managing the peers tracked in the peer store. It is RECOMMENDED that clients support:
automatic reconnection to peers under certain conditions connection keep-alive Reconnecting peers # A Waku v2 client MAY choose to reconnect to previously connected, managed peers under certain conditions. Such conditions include, but are not limited to:
Reconnecting to all relay-capable peers after a client restart. This will require persistent peer storage. If a client chooses to automatically reconnect to previous peers, it MUST respect the backing off period specified for GossipSub v1.1 before attempting to reconnect. This requires keeping track of the last time each peer was disconnected.
Connection keep-alive # A Waku v2 client MAY choose to implement a keep-alive mechanism to certain peers. If a client chooses to implement keep-alive on a connection, it SHOULD do so by sending periodic libp2p pings as per 10/WAKU2 client recommendations. The recommended period between pings SHOULD be at most 50% of the shortest idle connection timeout for the specific client and transport. For example, idle TCP connections often times out after 10 to 15 minutes.
Implementation note: the nim-waku client currently implements a keep-alive mechanism every 5 minutes, in response to a TCP connection timeout of 10 minutes.
Copyright # Copyright and related rights waived via CC0.
References # 10/WAKU2 client recommendations 11/WAKU2-RELAY 13/WAKU2-STORE libp2p peer ID libp2p ping protocol libp2p protocol IDs multiaddrs `}),e.add({id:21,href:"/spec/28/",title:"28/STATUS-FEATURING",section:"Docs",content:` Abstract # This specification describes a voting method to feature different active Status Communities.
Overview # When there is a active community that is seeking new members, current users of community should be able to feature their community so that it will be accessible to larger audience. Status community curation DApp should provide such a tool.
Rules of featuring: - Given community can\u0026rsquo;t be featured twice in a row. - Only one vote per user per community (single user can vote on multiple communities) - Voting will be done off-chain - If community hasn\u0026rsquo;t been featured votes for given community are still valid for the next 4 weeks
Since voting for featuring is similar to polling solutions proposed in this spec could be also used for different applications.
Voting # Voting for featuring will be done through waku v2.
Payload of waku message will be :
type FeatureVote = { voter: string // address of a voter sntAmount: BigNumber // amount of snt voted on featuring communityPK: string // public key of community timestamp: number // timestamp of message, must match timestamp of wakuMessage sign: string // cryptographic signature of a transaction (signed fields: voterAddress,sntAmount,communityPK,timestamp) } timestamp is necessary so that votes can\u0026rsquo;t be reused after 4 week period
Counting Votes # Votes will be counted by the DApp itself. DApp will aggregate all the votes in the last 4 weeks and calculate which communities should be displayed in the Featured tab of DApp.
Rules of counting: - When multiple votes from the same address on the same community are encountered only the vote with highest timestamp is considered valid. - If a community has been featured in a previous week it can\u0026rsquo;t be featured in current week. - In a current week top 5 (or 10) communities with highest amount of SNT votes up to previous Sunday 23:59:59 UTC are considered featured.
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:22,href:"/spec/29/",title:"29/WAKU2-CONFIG",section:"Docs",content:`29/WAKU2-CONFIG describes the RECOMMENDED values to assign to configurable parameters for Waku v2 clients. Since Waku v2 is built on libp2p, most of the parameters and reasonable defaults are derived from there.
Waku v2 relay messaging is specified in 11/WAKU2-RELAY, a minor extension of the libp2p GossipSub protocol. GossipSub behaviour is controlled by a series of adjustable parameters. Waku v2 clients SHOULD configure these parameters to the recommended values below.
GossipSub v1.0 parameters # GossipSub v1.0 parameters are defined in the corresponding libp2p specification. We repeat them here with RECOMMMENDED values for 11/WAKU2-RELAY implementations.
Parameter Purpose RECOMMENDED value D The desired outbound degree of the network 6 D_low Lower bound for outbound degree 4 D_high Upper bound for outbound degree 12 D_lazy (Optional) the outbound degree for gossip emission D heartbeat_interval Time between heartbeats 1 second fanout_ttl Time-to-live for each topic\u0026rsquo;s fanout state 60 seconds mcache_len Number of history windows in message cache 5 mcache_gossip Number of history windows to use when emitting gossip 3 seen_ttl Expiry time for cache of seen message ids 2 minutes GossipSub v1.1 parameters # GossipSub v1.1 extended GossipSub v1.0 and introduced several new parameters. We repeat the global parameters here with RECOMMMENDED values for 11/WAKU2-RELAY implementations.
Parameter Description RECOMMENDED value PruneBackoff Time after pruning a mesh peer before we consider grafting them again. 1 minute FloodPublish Whether to enable flood publishing true GossipFactor % of peers to send gossip to, if we have more than D_lazy available 0.25 D_score Number of peers to retain by score when pruning from oversubscription D_low D_out Number of outbound connections to keep in the mesh. D_low - 1 11/WAKU2-RELAY clients SHOULD implement a peer scoring mechanism with the parameter constraints as specified by libp2p.
Other configuration # The following behavioural parameters are not specified by libp2p, but nevertheless describes constraints that 11/WAKU2-RELAY clients MAY choose to implement.
Parameter Description RECOMMENDED value BackoffSlackTime Slack time to add to prune backoff before attempting to graft again 2 seconds IWantPeerBudget Maximum number of IWANT messages to accept from a peer within a heartbeat 25 IHavePeerBudget Maximum number of IHAVE messages to accept from a peer within a heartbeat 10 IHaveMaxLength Maximum number of messages to include in an IHAVE message 5000 Copyright # Copyright and related rights waived via CC0.
References # 11/WAKU2-RELAY GossipSub v1.0 parameters GossipSub v1.1 parameters libp2p libp2p GossipSub protocol `}),e.add({id:23,href:"/spec/3/",title:"3/REMOTE-LOG",section:"Docs",content:`A remote log is a replication of a local log. This means a node can read data that originally came from a node that is offline.
This specification is complemented by a proof of concept implementation1.
Definitions # Term Definition CAS Content-addressed storage. Stores data that can be addressed by its hash. NS Name system. Associates mutable data to a name. Remote log Replication of a local log at a different location. Wire Protocol # Secure Transport, storage, and name system # This specification does not define anything related to: secure transport, content addressed storage, or the name system. It is assumed these capabilities are abstracted away in such a way that any such protocol can easily be implemented.
Payloads # Payloads are implemented using protocol buffers v3.
CAS service:
syntax = \u0026#34;proto3\u0026#34;; package vac.cas; service CAS { rpc Add(Content) returns (Address) {} rpc Get(Address) returns (Content) {} } message Address { bytes id = 1; } message Content { bytes data = 1; } NS service:
syntax = \u0026#34;proto3\u0026#34;; package vac.cas; service NS { rpc Update(NameUpdate) returns (Response) {} rpc Fetch(Query) returns (Content) {} } message NameUpdate { string name = 1; bytes content = 2; } message Query { string name = 1; } message Content { bytes data = 1; } message Response { bytes data = 1; } Remote log:
syntax = \u0026#34;proto3\u0026#34;; package vac.cas; message RemoteLog { repeated Pair pair = 1; bytes tail = 2; message Pair { bytes remoteHash = 1; bytes localHash = 2; bytes data = 3; } } Synchronization # Roles # There are four fundamental roles:
Alice Bob Name system (NS) Content-addressed storage (CAS) The remote log protobuf is what is stored in the name system.
\u0026ldquo;Bob\u0026rdquo; can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.
Flow # Remote log # The remote log lets receiving nodes know what data they are missing. Depending on the specific requirements and capabilities of the nodes and name system, the information can be referred to differently. We distinguish between three rough modes:
Fully replicated log Normal sized page with CAS mapping \u0026ldquo;Linked list\u0026rdquo; mode - minimally sized page with CAS mapping Data format:
| H1_3 | H2_3 | | H1_2 | H2_2 | | H1_1 | H2_1 | | ------------| | next_page | Here the upper section indicates a list of ordered pairs, and the lower section contains the address for the next page chunk. H1 is the native hash function, and H2 is the one used by the CAS. The numbers corresponds to the messages.
To indicate which CAS is used, a remote log SHOULD use a multiaddr.
Embedded data:
A remote log MAY also choose to embed the wire payloads that corresponds to the native hash. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.
| H1_3 | | C_3 | | H1_2 | | C_2 | | H1_1 | | C_1 | | -------------| | next_page | Here C stands for the content that would be stored at the CAS.
Both patterns can be used in parallel, e,g. by storing the last k messages directly and use CAS pointers for the rest. Together with the next_page page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.
Next page semantics # The pointer to the \u0026rsquo;next page\u0026rsquo; is another remote log entry, at a previous point in time.
Interaction with MVDS # vac.mvds.Message payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.
Acknowledgments # TBD.
Copyright # Copyright and related rights waived via CC0.
Footnotes # https://github.com/vacp2p/research/tree/master/remote_log\u0026#160;\u0026#x21a9;\u0026#xfe0e;
`}),e.add({id:24,href:"/spec/30/",title:"30/ADAPTIVE-NODES",section:"Docs",content:`This is an informational spec that show cases the concept of adaptive nodes.
Node types - a continuum # We can look at node types as a continuum, from more restricted to less restricted, fewer resources to more resources.
Possible limitations # Connectivity: Not publicly connectable vs static IP and DNS Connectivity: Mostly offline to mostly online to always online Resources: Storage, CPU, Memory, Bandwidth Accessibility and motivation # Some examples:
Opening browser window: costs nothing, but contribute nothing Desktop: download, leave in background, contribute somewhat Cluster: expensive, upkeep, but can contribute a lot These are also illustrative, so a node in a browser in certain environment might contribute similarly to Desktop.
Adaptive nodes # We call these nodes adaptive nodes to highlights different modes of contributing, such as:
Only leeching from the network Relaying messages for one or more topics Providing services for lighter nodes such as lightpush and filter Storing historical messages to various degrees Ensuring relay network can\u0026rsquo;t be spammed with RLN Planned incentives # Incentives to run a node is currently planned around:
SWAP for accounting and settlement of services provided RLN RELAY for spam protection Other incentivization schemes are likely to follow and is an area of active research Node protocol selection # Each node can choose which protocols to support, depending on its resources and goals.
In the case of protocols like 11/WAKU2-RELAY etc (12, 13, 19, 21) these correspond to Libp2p protocols.
However, other protocols like 16/WAKU2-RPC (local HTTP JSON-RPC), 25/LIBP2P-DNS-DISCOVERY, Discovery v5 (DevP2P) or interfacing with distributed storage, are running on different network stacks.
This is in addition to protocols that specify payloads, such as 14/WAKU2-MESSAGE, 26/WAKU2-PAYLOAD, or application specific ones. As well as specs that act more as recommendations, such as 23/WAKU2-TOPICS or 27/WAKU2-PEERS.
Waku network visualization # We can better visualize the network with some illustrative examples.
Topology and topics # The first one shows an example topology with different PubSub topics for the relay protocol.
Legend # The dotted box shows what content topics (application-specific) a node is interested in.
A node that is purely providing a service to the network might not care.
In this example, we see support for toy chat, a topic in Waku v1 (Status chat), WalletConnect, and SuperRare community.
Auxiliary network # This is a separate component with its own topology.
Behavior and interaction with other protocols specified in Vac RFCs, e.g. 25/LIBP2P-DNS-DISCOVERY, 15/WAKU-BRIDGE, etc.
Node Cross Section # This one shows a cross-section of nodes in different dimensions and shows how the connections look different for different protocols.
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:25,href:"/spec/31/",title:"31/WAKU2-ENR",section:"Docs",content:` Abstract # This RFC describes the usage of the ENR (Ethereum Node Records) format for 10/WAKU2 purposes. The ENR format is defined in EIP-778 [3].
This RFC is an extension of EIP-778, ENR used in Waku v2 MUST adhere to both EIP-778 and 31/WAKU2-ENR.
Motivation # EIP-1459 with the usage of ENR has been implemented [1] [2] as a discovery protocol for Waku v2.
EIP-778 specifies a number of pre-defined keys. However, the usage of these keys alone does not allow for certain transport capabilities to be encoded, such as Websocket. Currently, Waku v2 nodes running in a Browser only support websocket transport protocol. Hence, new ENR keys need to be defined to allow for the encoding of transport protocol other than raw TCP.
Usage of Multiaddr Format Rationale # One solution would be to define new keys such as ws to encode the websocket port of a node. However, we expect new transport protocols to be added overtime such as quic. Hence, this would only provide a short term solution until another RFC would need to be added.
Moreover, secure websocket involves SSL certificates. SSL certificates are only valid for a given domain and ip, so an ENR containing the following information:
secure websocket port ipv4 fqdn ipv4 address ipv6 address Would carry some ambiguity: Is the certificate securing the websocket port valid for the ipv4 fqdn? the ipv4 address? the ipv6 address?
The 10/WAKU2 protocol family is built on the libp2p protocol stack. Hence, it uses multiaddr to format network addresses.
Directly storing one or several multiaddresses in the ENR would fix the issues listed above:
multiaddr is self-describing and support addresses for any network protocol: No new RFC would be needed to support encoding other transport protocols in an ENR. multiaddr contains both the host and port information, allowing the ambiguity previously described to be resolved. multiaddrs ENR key # We define a multiaddrs key.
The value MUST be a list of binary encoded multiaddr prefixed by their size. The size of the multiaddr MUST be encoded in a Big Endian unsigned 16-bit integer. The size of the multiaddr MUST be encoded in 2 bytes. The secp256k1 value MUST be present on the record; secp256k1 is defined in EIP-778 and contains the compressed secp256k1 public key. The node\u0026rsquo;s peer id SHOULD be deduced from the secp256k1 value. The multiaddresses SHOULD NOT contain a peer id except for circuit relay addresses For raw TCP \u0026amp; UDP connections details, EIP-778 pre-defined keys SHOULD be used; The keys tcp, udp, ip (and tcp6, udp6, ip6 for IPv6) are enough to convey all necessary information; To save space, multiaddrs key SHOULD only be used for connection details that cannot be represented using the EIP-778 pre-defined keys. The 300 bytes size limit as defined by EIP-778 still applies; In practice, it is possible to encode 3 multiaddresses in ENR, more or less could be encoded depending on the size of each multiaddress. Usage # Many connection types # Alice is a node operator, she runs a node that supports inbound connection for the following protocols:
TCP 10101 on 1.2.3.4 UDP 20202 on 1.2.3.4 TCP 30303 on 1234:5600:101:1::142 UDP 40404 on 1234:5600:101:1::142 Secure Websocket on wss://example.com:443/ QUIC on quic://quic.example.com:443/ A circuit relay address /ip4/1.2.3.4/tcp/55555/p2p/QmRelay/p2p-circuit/p2p/QmAlice Alice SHOULD structure the ENR for her node as follows:
key value tcp 10101 udp 20202 tcp6 30303 udp6 40404 ip 1.2.3.4 ip6 1234:5600:101:1::142 secp256k1 Alice\u0026rsquo;s compressed secp256k1 public key, 33 bytes multiaddrs len1 | /dns4/example.com/tcp/443/wss | len2 | /dns4/quic.examle.com/tcp/443/quic | len3 | /ip4/1.2.3.4/tcp/55555/p2p/QmRelay Where:
| is the concatenation operator, len1 is the length of /dns4/example.com/tcp/443/wss byte representation, len2 is the length of /dns4/quic.examle.com/tcp/443/quic byte representation. len3 is the length of /ip4/1.2.3.4/tcp/55555/p2p/QmRelay byte representation. Notice that the /p2p-circuit component is not stored, but, since circuit relay addresses are the only one containing a p2p component, it\u0026rsquo;s safe to assume that any address containing this component is a circuit relay address. Decoding this type of multiaddresses would require appending the /p2p-circuit component. Raw TCP only # Bob is a node operator, he runs a node that supports inbound connection for the following protocols:
TCP 10101 on 1.2.3.4 Bob SHOULD structure the ENR for her node as follows:
key value tcp 10101 ip 1.2.3.4 secp256k1 Bob\u0026rsquo;s compressed secp256k1 public key, 33 bytes Indeed, as Bob\u0026rsquo;s node\u0026rsquo;s connection details can be represented with EIP-778\u0026rsquo;s pre-defined keys only then it is not needed to use the multiaddrs key.
Limitations # Supported key type is secp256k1 only.
In the future, an extension of this RFC could be made to support other elliptic curve cryptography such as ed25519.
waku2 ENR key # We define a waku2 field key:
The value MUST be an 8-bit flag field, where bits set to 1 indicate true and bits set to 0 indicate false for the relevant flags. The flag values already defined are set out below, with bit 7 the most significant bit and bit 0 the least significant bit. bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 undef undef undef undef lightpush filter store relay In the scheme above, the flags lightpush, filter, store and relay correlates with support for protocols with the same name. If a protocol is not supported, the corresponding field MUST be set to false. Indicating positive support for any specific protocol is OPTIONAL, though it MAY be required by the relevant application or discovery process. Flags marked as undef is not yet defined. These SHOULD be set to false by default. Usage # A Waku v2 node MAY choose to populate the waku2 field for enhanced discovery capabilities, such as indicating supported protocols. Such a node MAY indicate support for any specific protocol by setting the corresponding flag to true. Waku v2 nodes that want to participate in Node Discovery Protocol v5 [4], however, MUST implement the waku2 key with at least one flag set to true. Waku v2 nodes that discovered other participants using Discovery v5, MUST filter out participant records that do not implement this field or do not have at least one flag set to true. In addition, such nodes MAY choose to filter participants on specific flags (such as supported protocols), or further interpret the waku2 field as required by the application. Copyright # Copyright and related rights waived via CC0.
References # [1] https://github.com/status-im/nim-waku/pull/690 [2] https://github.com/vacp2p/rfc/issues/462#issuecomment-943869940 [3] https://eips.ethereum.org/EIPS/eip-778 [4] https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md `}),e.add({id:26,href:"/spec/32/",title:"32/RLN-V1",section:"Docs",content:` Abstract # The following specification covers the RLN construct as well as some auxiliary libraries useful for interacting with it. Rate limiting nullifier (RLN) is a construct based on zero-knowledge proofs that provides an anonymous rate-limited signaling/messaging framework suitable for decentralized (and centralized) environments. Anonymity refers to the unlinkability of messages to their owner.
Motivation # RLN guarantees a messaging rate is enforced cryptographically while preserving the anonymity of the message owners. A wide range of applications can benefit from RLN and provide desirable security features. For example, an e-voting system can integrate RLN to contain the voting rate while protecting the voters-vote unlinkability. Another use case is to protect an anonymous messaging system against DDoS and spam attacks by containing messaging rate of users. This latter use case is explained in 17/WAKU2-RLN-RELAY RFC.
Flow # The users participate in the protocol by first registering to an application-defined group referred by the membership group. Registration to the group is mandatory for signaling in the application. After registration, group members can generate Zero-knowledge Proof of membership for their signals and can participate in the application. Usually, the membership requires a financial or social stake which is beneficial for the prevention of Sybil attacks and double-signaling. Group members are allowed to send one signal per external nullifier (an identifier that groups signals and can be thought of as a voting booth). If a user generates more signals than allowed, the user risks being slashed - by revealing his membership secret credentials. If the financial stake is put in place, the user also risks his stake being taken.
Generally the flow can be described by the following steps:
Registration Signaling Verification and slashing Registration # Depending on the application requirements, the registration can be implemented in different ways, for example:
centralized registrations, by using a central server decentralized registrations, by using a smart contract What is important is that the users\u0026rsquo; identity commitments (explained in section User Indetity) are stored in a Merkle tree, and the users can obtain a Merkle proof proving that they are part of the group.
Also depending on the application requirements, usually a financial or social stake is introduced.
An example for financial stake is: For each registration a certain amount of ETH is required. An example for social stake is using InterRep as a registry - users need to prove that they have a highly reputable social media account.
Implementation notes # User identity # The user\u0026rsquo;s identity is composed of:
{ identity_secret: [identity_nullifier, identity_trapdoor], identity_secret_hash: poseidonHash(identity_secret), identity_commitment: poseidonHash([identity_secret_hash]) } For registration, the user needs to submit their identity_commitment (along with any additional registration requirements) to the registry. Upon registration, they should receive leaf_index value which represents their position in the Merkle tree. Receiving a leaf_index is not a hard requirement and is application specific. The other way around is the users calculating the leaf_index themselves upon successful registration.
Signaling # After registration, the users can participate in the application by sending signals to the other participants in a decentralised manner or to a centralised server. Along with their signal, they need to generate a ZK-Proof by using the circuit with the specification described above.
For generating a proof, the users need to obtain the required parameters or compute them themselves, depending on the application implementation and client libraries supported by the application. For example the users can store the membership Merkle tree on their end and generate a Merkle proof whenever they want to generate a signal.
Implementation notes # Signal hash # The signal hash can be generated by hashing the raw signal (or content) using the keccak256 hash function.
External nullifier # The external nullifier MUST be computed as the Poseidon hash of the current epoch (e.g. a value equal to or derived from the current UNIX timestamp divided by the epoch length) and the RLN identifier.
external_nullifier = poseidonHash([epoch, rln_identifier]) Obtaining Merkle proof # The Merkle proof should be obtained locally or from a trusted third party. By using the incremental Merkle tree algorithm, the Merkle can be obtained by providing the leaf_index of the identity_commitment. The proof (Merkle_proof) is composed of the following fields:
{ root: bigint indices: number[] path_elements: bigint[][] } root - The root of membership group Merkle tree at the time of publishing the message indices - The index fields of the leafs in the Merkle tree - used by the Merkle tree algorithm for verification path_elements - Auxiliary data structure used for storing the path to the leaf - used by the Merkle proof algorithm for verificaton Generating proof # For proof generation, the user need to submit the following fields to the circuit:
{ identity_secret: identity_secret_hash, path_elements: Merkle_proof.path_elements, identity_path_index: Merkle_proof.indices, x: signal_hash, external_nullifier: external_nullifier } Calculating output # The proof output is calculated locally, in order for the required fields for proof verification to be sent along with the proof. The proof output is composed of the y share of the secret equation and the internal_nullifier. The internal_nullifier represents a unique fingerprint of a user for a given epoch and app. The following fields are needed for proof output calculation:
{ identity_secret_hash: bigint, external_nullifier: bigint, x: bigint, } The output [y, internal_nullifier] is calculated in the following way:
a_0 = identity_secret_hash a_1 = poseidonHash([a0, external_nullifier]) y = a_0 + x * a_1 internal_nullifier = poseidonHash([a_1]) It relies on the properties of the Shamir\u0026rsquo;s Secret sharing scheme.
Sending the output message # The user\u0026rsquo;s output message (output_message), containing the signal should contain the following fields at minimum:
{ signal: signal, # non-hashed signal proof: zk_proof, internal_nullifier: internal_nullifier, x: x, # signal_hash y: y, rln_identifier: rln_identifier } Additionally depending on the application, the following fields might be required:
{ root: Merkle_proof.root, epoch: epoch } Verification and slashing # The slashing implementation is dependent on the type of application. If the application is implemented in a centralised manner, and everything is stored on a single server, the slashing will be implemented only on the server. Otherwise if the application is distributed, the slashing will be implemented on each user\u0026rsquo;s client.
Implementation notes # Each user of the protocol (server or otherwise) will need to store metadata for each message received by each user, for the given epoch. The data can be deleted when the epoch passes. Storing metadata is required, so that if a user sends more than one unique signal per epoch, they can be slashed and removed from the protocol. The metadata stored contains the x, y shares and the internal_nullifier for the user for each message. If enough such shares are present, the user\u0026rsquo;s secret can be retreived.
One way of storing received metadata (messaging_metadata) is the following format:
{ [external_nullifier]: { [internal_nullifier]: { x_shares: [], y_shares: [] } } } Verification # The output message verification consists of the following steps:
external_nullifier correctness non-duplicate message check zk_proof verification spam verification 1. external_nullifier correctness Upon received output_message, first the epoch and rln_identifier fields are checked, to ensure that the message matches the current external_nullifier. If the external_nullifier is correct the verification continues, otherwise, the message is discarded.
2. non-duplicate message check The received message is checked to ensure it is not duplicate. The duplicate message check is performed by verifying that the x and y fields do not exist in the messaging_metadata object. If the x and y fields exist in the x_shares and y_shares array for the external_nullifier and the internal_nullifier the message can be considered as a duplicate. Duplicate messages are discarded.
3. zk_proof verification
The zk_proof should be verified by providing the zk_proof field to the circuit verifier along with the public_signal:
[ y, Merkle_proof.root, internal_nullifier, x, # signal_hash external_nullifier ] If the proof verification is correct, the verification continues, otherwise the message is discarded.
4. Double signaling verification
After the proof is verified the x, and y fields are added to the x_shares and y_shares arrays of the messaging_metadata external_nullifier and internal_nullifier object. If the length of the arrays is equal to the signaling threshold (limit), the user can be slashed.
Slashing # After the verification, the user can be slashed if two different shares are present to reconstruct their identity_secret_hash from x_shares and y_shares fields, for their internal_nullifier. The secret can be retreived by the properties of the Shamir\u0026rsquo;s secret sharing scheme. In particular the secret (a_0) can be retrieved by computing Lagrange polynomials.
After the secret is retreived, the user\u0026rsquo;s identity_commitment can be generated from the secret and it can be used for removing the user from the membership Merkle tree (zeroing out the leaf that contains the user\u0026rsquo;s identity_commitment). Additionally, depending on the application the identity_secret_hash can be used for taking the user\u0026rsquo;s provided stake.
Technical overview # The main RLN construct is implemented using a ZK-SNARK circuit. However, it is helpful to describe the other necessary outside components for interaction with the circuit, which together with the ZK-SNARK circuit enable the above mentioned features.
Terminology # Term Description ZK-SNARK https://z.cash/technology/zksnarks/ Stake Financial or social stake required for registering in the RLN applications. Common stake examples are: locking cryptocurrency (financial), linking reputable social identity. Identity secret An array of two unique random components (identity nullifier and identity trapdoor), which must be kept private by the user. Secret hash and identity commitment are derived from this array. Identity nullifier Random 32 byte value used as component for identity secret generation. Identity trapdoor Random 32 byte value used as component for identity secret generation. Identity secret hash The hash of the identity secret, obtained using the Poseidon hash function. It is used for deriving the identity commitment of the user, and as a private input for zk proof generation. The secret hash should be kept private by the user. Identity commitment Hash obtained from the Identity secret hash by using the poseidon hash function. It is used by the users for registering in the protocol. Signal The message generated by a user. It is an arbitrary bit string that may represent a chat message, a URL request, protobuf message, etc. Signal hash Keccak256 hash of the signal modulo circuit\u0026rsquo;s field characteristic, used as an input in the RLN circuit. RLN Identifier Random finite field value unique per RLN app. It is used for additional cross-application security. The role of the RLN identifier is protection of the user secrets from being compromised when signals are being generated with the same credentials in different apps. RLN membership tree Merkle tree data structure, filled with identity commitments of the users. Serves as a data structure that ensures user registrations. Merkle proof Proof that a user is member of the RLN membership tree. RLN ZK-Circuit specific terms # Term Description x Keccak hash of the signal, same as signal hash (Defined above). A0 The identity secret hash. A1 Poseidon hash of [A0, External nullifier] (see about External nullifier below). y The result of the polynomial equation (y = a0 + a1*x). The public output of the circuit. External nullifier Poseidon hash of [Epoch, RLN Identifier]. An identifier that groups signals and can be thought of as a voting booth. Internal nullifier Poseidon hash of [A1]. This field ensures that a user can send only one valid signal per external nullifier without risking being slashed. Public input of the circuit. ZK Circuits specification # Anonymous signaling with a controlled rate limit is enabled by proving that the user is part of a group which has high barriers to entry (form of stake) and enabling secret reveal if more than 1 unique signal is produced per external nullifier. The membership part is implemented using membership Merkle trees and Merkle proofs, while the secret reveal part is enabled by using the Shamir\u0026rsquo;s Secret Sharing scheme. Essentially the protocol requires the users to generate zero-knowledge proof to be able to send signals and participate in the application. The zero knowledge proof proves that the user is member of a group, but also enforces the user to share part of their secret for each signal in an external nullifier. The external nullifier is usually represented by timestamp or a time interval. It can also be thought of as a voting booth in voting applications.
The ZK Circuit is implemented using a Groth-16 ZK-SNARK, using the circomlib library.
System parameters # DEPTH - Merkle tree depth Circuit parameters # Public Inputs
x external_nullifier Private Inputs
identity_secret_hash path_elements - rln membership proof component identity_path_index - rln membership proof component Outputs
y root - the rln membership tree root internal_nullifier Hash function # Canonical Poseidon hash implementation is used, as implemented in the circomlib library, according to the Poseidon paper. This Poseidon hash version (canonical implementation) uses the following parameters:
Hash inputs t RF RP 1 2 8 56 2 3 8 57 3 4 8 56 4 5 8 60 5 6 8 60 6 7 8 63 7 8 8 64 8 9 8 63 Membership implementation # For a valid signal, a user\u0026rsquo;s identity_commitment (more on identity commitments below) must exist in identity membership tree. Membership is proven by providing a membership proof (witness). The fields from the membership proof required for the verification are: path_elements and identity_path_index.
IncrementalQuinTree algorithm is used for constructing the Membership Merkle tree. The circuits are reused from this repository. You can find out more details about the IncrementalQuinTree algorithm here.
Slashing and Shamir\u0026rsquo;s Secret Sharing # Slashing is enabled by using polynomials and Shamir\u0026rsquo;s Secret sharing. In order to produce a valid proof, identity_secret_hash as a private input to the circuit. Then a secret equation is created in the form of:
y = a_0 + x * a_1, where a_0 is the identity_secret_hash and a_1 = hash(a_0, external nullifier). Along with the generated proof, the users need to provide a (x, y) share which satisfies the line equation, in order for their proof to be verified. x is the hashed signal, while the y is the circuit output. With more than one pair of unique shares, anyone can derive a_0, i.e. the identity_secret_hash . The hash of a signal will be the evaluation point x. In this way, a member who sends more than one unique signal per external_nullifier risks their identity secret being revealed.
Note that shares used in different epochs and different RLN apps cannot be used to derive the identity secret hash.
Thanks to the external_nullifier definition, also shares computed from same secret within same epoch but in different RLN apps cannot be used to derive the identity secret hash.
The rln_identifier is a random value from a finite field, unique per RLN app, and is used for additional cross-application security - to protect the user secrets being compromised if they use the same credentials accross different RLN apps. If rln_identifier is not present, the user uses the same credentials and sends a different message for two different RLN apps using the same external_nullifier, then their user signals can be grouped by the internal_nullifier which could lead the user\u0026rsquo;s secret revealed. This is because two separate signals under the same internal_nullifier can be treated as rate limiting violation. With adding the rln_identifier field we obscure the internal_nullifier, so this kind of attack can be hardened because we don\u0026rsquo;t have the same internal_nullifier anymore.
Identity credentials generation # In order to be able to generate valid proofs, the users need to be part of the identity membership Merkle tree. They are part of the identity membership Merkle tree if their identity_commitment is placed in a leaf in the tree.
The identity credentials of a user are composed of:
identity_secret identity_secret_hash identity_commitment identity_secret # The identity_secret is generated in the following way:
identity_nullifier = random_32_byte_buffer identity_trapdoor = random_32_byte_buffer identity_secret = [identity_nullifier, identity_trapdoor] The same secret should not be used accross different protocols, because revealing the secret at one protocol could break privacy for the user in the other protocols.
identity_secret_hash # The identity_secret_hash is generated by obtaining a Poseidon hash of the identity_secret array:
identity_secret_hash = poseidonHash(identity_secret) identity_commitment # The identity_commitment is generated by obtaining a Poseidon hash of the identity_secret_hash:
identity_commitment = poseidonHash([identity_secret_hash]) Appendix A: Security considerations # RLN is an experimental and still un-audited technology. This means that the circuits have not been yet audited. Another consideration is the security of the underlying primitives. zk-SNARKS require a trusted setup for generating a prover and verifier keys. The standard for this is to use trusted Multi-Party Computation (MPC) ceremony, which requires two phases. Trusted MPC ceremony has not yet been performed for the RLN circuits.
SSS security assumptions # Shamir-Secret Sharing requires polynomial coefficients to be independent of each other. However, a_1 depends on a_0 through the Poseidon hash algorithm. Due to the design of Poseidon, it is possible to attack the protocol.
It was decided not to change the circuits design, since at the moment the attack is infeasible. Therefore, implementers must be aware that the current version provides approximately 160-bit security and not 254. Possible improvements:
change the circuit to make coefficients independent; switch to other hash function (Keccak, SHA); Appendix B: Identity scheme choice # The hashing scheme used is based on the design decisions which also include the Semaphore circuits. Our goal was to ensure compatibility of the secrets for apps that use Semaphore and RLN circuits while also not compromising on security because of using the same secrets.
For example let\u0026rsquo;s say there is a voting app that uses Semaphore, and also a chat app that uses RLN. The UX would be better if the users would not need to care about complicated identity management (secrets and commitments) t hey use for each app, and it would be much better if they could use a single id commitment for this. Also in some cases these kind of dependency is required - RLN chat app using Interep as a registry (instead of using financial stake). One potential concern about this interoperability is a slashed user on the RLN app side having their security compromised on the semaphore side apps as well. I.e obtaining the user\u0026rsquo;s secret, anyone would be able to generate valid semaphore proofs as the slashed user. We don\u0026rsquo;t want that, and we should keep user\u0026rsquo;s app specific security threats in the domain of that app alone.
To achieve the above interoperability UX while preventing the shared app security model (i.e slashing user on an RLN app having impact on Semaphore apps), we had to do the follow in regard the identity secret and identity commitment:
identity_secret = [identity_nullifier, identity_trapdoor] identity_secret_hash = poseidonHash(identity_secret) identity_commitment = poseidonHash([identity_secret_hash]) Secret components for generating Semaphore proof:
identity_nullifier identity_trapdoor Secret components for generting RLN proof:
identity_secret_hash When a user is slashed on the RLN app side, their identity secret hash is revealed. However a semaphore proof can\u0026rsquo;t be generated because we do not know the user\u0026rsquo;s nullifier and trapdoor.
With this design we achieve:
identity commitment (Semaphore) == identity commitment (RLN) secret (semaphore) != secret (RLN).
This is the only option we had for the scheme in order to satisfy the properties described above.
Also for RLN we do a single secret component input for the circuit. Thus we need to hash the secret array (two components) to a secret hash, and we use that as a secret component input.
Appendix C: Auxiliary tooling # There are few additional tools implemented for easier integrations and usage of the RLN protocol.
zerokit is a set of Zero Knowledge modules, written in Rust and designed to be used in many different environments. Among different modules, it supports Semaphore and RLN.
zk-kit is a typescript library which exposes APIs for identity credentials generation, as well as proof generation. It supports various protocols (Semaphore, RLN).
zk-keeper is a browser plugin which allows for safe credential storing and proof generation. You can think of MetaMask for ZK-Proofs. It uses zk-kit under the hood.
Appendix D: Example usage # The following examples are code snippets using the zerokit RLN module. The examples are written in rust.
Creating a RLN object # use rln::protocol::*; use rln::public::*; use std::io::Cursor; // We set the RLN parameters: // - the tree height; // - the circuit resource folder (requires a trailing \u0026#34;/\u0026#34;). let tree_height = 20; let resources = Cursor::new(\u0026#34;../zerokit/rln/resources/tree_height_20/\u0026#34;); // We create a new RLN instance let mut rln = RLN::new(tree_height, resources); Generating identity credentials # // We generate an identity tuple let mut buffer = Cursor::new(Vec::\u0026lt;u8\u0026gt;::new()); rln.extended_key_gen(\u0026amp;mut buffer).unwrap(); // We deserialize the keygen output to obtain // the identiy_secret and id_commitment let (identity_trapdoor, identity_nullifier, identity_secret_hash, id_commitment) = deserialize_identity_tuple(buffer.into_inner()); Adding ID commitment to the RLN Merkle tree # // We define the tree index where id_commitment will be added let id_index = 10; // We serialize id_commitment and pass it to set_leaf let mut buffer = Cursor::new(serialize_field_element(id_commitment)); rln.set_leaf(id_index, \u0026amp;mut buffer).unwrap(); Setting epoch and signal # // We generate epoch from a date seed and we ensure is // mapped to a field element by hashing-to-field its content let epoch = hash_to_field(b\u0026#34;Today at noon, this year\u0026#34;); // We set our signal let signal = b\u0026#34;RLN is awesome\u0026#34;; Generating proof # // We prepare input to the proof generation routine let proof_input = prepare_prove_input(identity_secret, id_index, epoch, signal); // We generate a RLN proof for proof_input let mut in_buffer = Cursor::new(proof_input); let mut out_buffer = Cursor::new(Vec::\u0026lt;u8\u0026gt;::new()); rln.generate_rln_proof(\u0026amp;mut in_buffer, \u0026amp;mut out_buffer) .unwrap(); // We get the public outputs returned by the circuit evaluation let proof_data = out_buffer.into_inner(); Verifiying proof # // We prepare input to the proof verification routine let verify_data = prepare_verify_input(proof_data, signal); // We verify the zk-proof against the provided proof values let mut in_buffer = Cursor::new(verify_data); let verified = rln.verify(\u0026amp;mut in_buffer).unwrap(); // We ensure the proof is valid assert!(verified); For more details please visit the zerokit library.
Copyright # Copyright and related rights waived via CC0
References # [1] https://medium.com/privacy-scaling-explorations/rate-limiting-nullifier-a-spam-protection-mechanism-for-anonymous-environments-bbe4006a57d [2] https://github.com/appliedzkp/zk-kit [3] https://github.com/akinovak/zk-keeper [4] https://z.cash/technology/zksnarks/ [5] https://en.wikipedia.org/wiki/Merkle_tree [6] https://eprint.iacr.org/2016/260.pdf [7] https://docs.circom.io/ [8] https://eprint.iacr.org/2019/458.pdf [9] https://github.com/appliedzkp/incrementalquintree [10] https://ethresear.ch/t/gas-and-circuit-constraint-benchmarks-of-binary-and-quinary-incremental-merkle-trees-using-the-poseidon-hash-function/7446 [11] https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing [12] https://research.nccgroup.com/2020/06/24/security-considerations-of-zk-snark-parameter-multi-party-computation/ [13] https://github.com/Rate-Limiting-Nullifier/rln-circuits/ [14] https://rate-limiting-nullifier.github.io/rln-docs/ `}),e.add({id:27,href:"/spec/33/",title:"33/WAKU2-DISCV5",section:"Docs",content:` Abstract # 33/WAKU2-DISCV5 specifies a modified version of Ethereum\u0026rsquo;s Node Discovery Protocol v5 as a means for ambient node discovery. 10/WAKU2 uses the 33/WAKU2-DISCV5 ambient node discovery network for establishing a decentralized network of interconnected Waku2 nodes. In its current version, the 33/WAKU2-DISCV5 discovery network is isolated from the Ethereum Discovery v5 network. Isolation improves discovery efficiency, which is especially significant with a low number of Waku nodes compared to the total number of Ethereum nodes.
Disclaimer # This version of 33/WAKU2-DISCV5 has a focus on timely deployment of an efficient discovery method for 10/WAKU2. Establishing a separate discovery network is in line with this focus. However, we are aware of potential resilience problems (see section on security considerations) and are discussing and researching hybrid approaches.
Background and Rationale # 11/WAKU2-RELAY assumes the existence of a network of Waku2 nodes. For establishing and growing this network, new nodes trying to join the Waku2 network need a means of discovering nodes within the network. 10/WAKU2 supports the following discovery methods in order of increasing decentralization
hard coded bootstrap nodes DNS discovery (based on EIP-1459) peer-exchange (work in progress) 33/WAKU2-DISCV5 (specified in this document) The purpose of ambient node discovery within 10/WAKU2 is discovering Waku2 nodes in a decentralized way. The unique selling point of 33/WAKU2-DISCV5 is its holistic view of the network, which allows avoiding hotspots and allows merging the network after a split. While the other methods provide either a fixed or local set of nodes, 33/WAKU2-DISCV5 can provide a random sample of Waku2 nodes. Future iterations of this document will add the possibility of efficiently discovering Waku2 nodes that have certain capabilities, e.g. holding messages of a certain time frame during which the querying node was offline.
Separate Discovery Network # w.r.t. Waku2 Relay Network # 33/WAKU2-DISCV5 spans an overlay network separate from the GossipSub network 11/WAKU2-RELAY builds on. Because it is a P2P network on its own, it also depends on bootstrap nodes. Having a separate discovery network reduces load on the bootstrap nodes, because the actual work is done by randomly discovered nodes. This also increases decentralization.
w.r.t. Ethereum Discovery v5 # 33/WAKU2-DISCV5 spans a discovery network isolated from the Ethereum Discovery v5 network.
Another simple solution would be taking part in the Ethereum Discovery network, and filtering Waku nodes based on whether they support 31/WAKU2-ENR. This solution is more resilient towards eclipse attacks. However, this discovery method is very inefficient for small percentages of Waku nodes (see estimation). It boils down to random walk discovery and does not offer a O(log(n)) hop bound. The rarer the requested property (in this case Waku), the longer a random walk will take until finding an appropriate node, which leads to a needle-in-the-haystack problem. Using a dedicated Waku2 discovery network, nodes can query this discovery network for a random set of nodes and all (well-behaving) returned nodes can serve as bootstrap nodes for other Waku2 protocols.
A more sophisticated solution would be using Discv5 topic discovery. However, in its current state it also has efficiency problems for small percentages of Waku nodes and is still in the design phase (see here).
Currently, the Ethereum discv5 network is very efficient in finding other discv5 nodes, but it is not so efficient for finding discv5 nodes that have a specific property or offer specific services, e.g. Waku.
As part of our discv5 roadmap, we consider two ideas for future versions of 33/WAKU2-DISCV5
Discv5 topic discovery with adjustments (ideally upstream) a hybrid solution that uses both a separate discv5 network and a Waku-ENR-filtered Ethereum discv5 network Semantics # 33/WAKU2-DISCV5 fully inherits the discv5 semantics.
Before announcing their address via Waku2 discv5, nodes SHOULD check if this address is publicly reachable. Nodes MAY use the libp2p AutoNAT protocol to perform that check. Nodes SHOULD only announce publicly reachable addresses via Waku2 discv5, to avoid cluttering peer lists with nodes that are not reachable.
Wire Format Specification # 33/WAKU2-DISCV5 inherits the discv5 wire protocol except for the following differences
WAKU2-Specific protocol-id # Ethereum discv5:
33/WAKU2-DISCV5:
Suggestions for Implementations # Existing discv5 implementations
can be augmented to make the protocol-id selectable using a compile-time flag as in this feature branch of nim-eth/discv5. can be forked followed by changing the protocol-id string as in go-waku. Security Considerations # Sybil attack # Implementations should limit the number of bucket entries that have the same network parameters (IP address / port) to mitigate Sybil attacks.
Eclipse attack # Eclipse attacks aim to eclipse certain regions in a DHT. Malicious nodes provide false routing information for certain target regions. The larger the desired eclipsed region, the more resources (i.e. controlled nodes) the attacker needs. This introduces an efficiency versus resilience tradeoff. Discovery is more efficient if information about target objects (e.g. network parameters of nodes supporting Waku) are closer to a specific DHT address. If nodes providing specific information are closer to each other, they cover a smaller range in the DHT and are easier to eclipse.
Sybil attacks greatly increase the power of eclipse attacks, because they significantly reduce resources necessary to mount a successful eclipse attack.
Security Implications of a Separate Discovery Network # A dedicated Waku discovery network is more likely to be subject to successful eclipse attacks (and to DoS attacks in general). This is because eclipsing in a smaller network requires less resources for the attacker. DoS attacks render the whole network unusable if the percentage of attacker nodes is sufficient.
Using random walk discovery would mitigate eclipse attacks targeted at specific capabilities, e.g. Waku. However, this is because eclipse attacks aim at the DHT overlay structure, which is not used by random walks. So, this mitigation would come at the cost of giving up overlay routing efficiency. The efficiency loss is especially severe with a relatively small number of Waku nodes.
Properly protecting against eclipse attacks is challenging and raises research questions that we will address in future stages of our discv5 roadmap.
References # 10/WAKU2 11/WAKU2-RELAY 31/WAKU2-ENR Node Discovery Protocol v5 (discv5) discv5 semantics. discv5 wire protocol discv5 topic discovery Waku DNS discovery libp2p AutoNAT protocol EIP-1459 GossipSub Waku discv5 roadmap discussion discovery efficiency estimation implementation: Nim implementation: Go Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:28,href:"/spec/34/",title:"34/WAKU2-PEER-EXCHANGE",section:"Docs",content:` Abstract # This document specifies a simple request-response peer exchange protocol. Responders send information about a requested number of peers. The main purpose of this protocol is providing resource restricted devices with peers.
Protocol identifier: /vac/waku/peer-exchange/2.0.0-alpha1
Background and Motivation # It may not be feasible on resource restricted devices to take part in distributed random sampling ambient peer discovery protocols such as 33/WAKU2-DISCV5. The Waku peer discovery protocol specified in this document allows resource restricted devices to request a list of peers from a service node. Network parameters necessary to connect to this service node COULD be learned from a static bootstrapping method or using EIP-1459: Node Discovery via DNS. The advantage of using Waku peer exchange to discover new peers over using a static peer list or DNS discovery is a more even load distribution. If a lot of (resource restricted) nodes would use the same service nodes as relay or store nodes, the load on these would be very high. Heavily used static nodes also add a centralized element. Downtime of such a node might significantly impact the network.
However, the resource efficiency of this protocol comes at an anonymity cost, which is explained in the Security/Privacy Considerations section. This protocol SHOULD only be used if 33/WAKU2-DISCV5 is infeasible.
Theory and Protocol Semantics # The peer exchange protocol specified in this document is a simple request-response protocol. As Figure 1 illustrates, the requesting node sends a request to a peer, which acts as the responder. The responder replies with a list of ENRs as specified in 31/WAKU2-ENR. The multiaddresses used to connect to the respective peers can be extracted from the ENRs.
In order to protect its anonymity, the responder MUST NOT provide peers from its actively used peer list as this opens pathways to Neighbourhood Surveillance attacks, as described in the Security/Privacy Considerations Section. The responder SHOULD provide a set of peers that has been retrieved using ambient peer discovery methods supporting random sampling, e.g. 33/WAKU2-DISCV5. This both protects the responder\u0026rsquo;s anonymity as well as helps distributing load.
To allow for fast responses, responders SHOULD retrieve peers unsolicited (before receiving a query) and maintain a queue of peers for the purpose of providing them in peer exchange responses. To get the best anonymity properties with respect to response peer sets, responders SHOULD use each of these peers only once.
To save bandwidth, and as a trade off to anonymity, responders MAY maintain a larger cache of exchange peers and randomly sample response sets from this local cache. The size of the cache SHOULD be large enough to allow randomly sampling peer sets that (on average) do not overlap too much. The responder SHOULD periodically replace the oldest peers in the cache. This document provides recommended choices for the cache size in the Implementation Suggestions Section.
Requesters, in the context of the specified peer exchange protocol, SHOULD be resource restricted devices. While any node could technically act as a requester, using the peer exchange protocol comes with two drawbacks:
reducing anonymity causing load on responder nodes Wire Format Specification # syntax = \u0026#34;proto3\u0026#34;; message PeerInfo { bytes enr = 1; } message PeerExchangeQuery { uint64 num_peers = 1; } message PeerExchangeResponse { repeated PeerInfo peer_infos = 1; } message PeerExchangeRPC { PeerExchangeQuery query = 1; PeerExchangeResponse response = 2; } The enr field contains a Waku ENR as specified in 31/WAKU2-ENR.
Requesters send a PeerExchangeQuery to a peer. Responders SHOULD include a maximum of num_peers PeerInfo instances into a response. Responders send a PeerExchangeResponse to requesters containing a list of PeerInfo instances, which in turn hold an ENR.
Implementation Suggestions # Discovery Interface # Implementations can implement the libp2p discovery interface (e.g. nim, javascript).
Exchange Peer Cache Size # The size of the (optional) exchange peer cache discussed in Theory and Protocol Semantics depends on the average number of requested peers, which is expected to be the outbound degree of the underlying libp2p gossipsub mesh network. The recommended value for this outbound degree is 6 (see parameter D in 29/WAKU2-CONFIG). It is recommended for the cache to hold at least 10 times as many peers (60).
The recommended cache size also depends on the number of requesters a responder is expected to serve within a refresh cycle. A refresh cycle is the time interval in which all peers in the cache are expected to be replaced. If the number of requests expected per refresh cycle exceeds 600 (10 times the above recommended 60), it is recommended to increase the cache size to at least a tenth of that number.
We will investigate peer exchange cache sizes and refresh strategies, and provide suggestions based on that in future versions (draft, stable) of this document.
Security/Privacy/Anonymity Considerations # The peer exchange protocol specified in this document comes with anonymity and security implications. We differentiate these implications into the requester and responder side, respectively.
Requester # With a simple peer exchange protocol, the requester is inherently susceptible to both neighbourhood surveillance and controlled neighbourhood attacks.
To mount a neighbourhood surveillance attack, an attacker has to connect to the peers of the victim node. The peer exchange protocol allows a malicious responder to easily get into this position. The responder connects to a set of peers and simply returns this set of peers to the requester.
The peer exchange protocol also makes it much easier to get into the position required for the controlled neighbourhood attack: A malicious responder provides controlled peers in the response peer list.
More on these attacks may be found in our research log article.
As a weak mitigation the requester MAY ask several peers and select a subset of the returned peers.
Responder # Responders that answer with active mesh peers are more vulnerable to a neighbourhood surveillance attack. Responding with the set of active mesh peers allows a malicious requester to get into the required position more easily. It takes away the first hurdle of the neighbourhood surveillance attack: The attacker knows which peers to try to connect to. This increased vulnerability can be avoided by only responding with randomly sampled sets of peers, e.g. by requesting a random peer set via 33/WAKU2-DISCV5. (As stated in the Theory and Protocol Semantics Section, these peer sets SHOULD be retrieved unsolicitedly before receiving requests to achieve faster response times.)
Responders are also susceptible to amplification DoS attacks. Requesters send a simple message request which causes responders to engage in ambient peer discovery to retrieve a new random peer set. As a mitigation, responders MAY feature a seen cache for requests and only answer once per time interval. The exchange-peer cache discussed in Theory and Protocol Semantics Section also provides mitigation. Still, frequent queries can tigger the refresh cycle more often. The seen cache MAY be used in conjunction to provide additional mitigation.
Further Considerations # The response field contains ENRs as specified in 31/WAKU2-ENR. While ENRs contain signatures, they do not violate the Waku relay no-sign policy), because they are part of the discovery domain and are not propagated in the relay domain. However, there might still be some form of leakage: ENRs could be used to track peers and facilitate linking attacks. We will investigate this further in our Waku anonymity analysis.
Copyright # Copyright and related rights waived via CC0.
References # 33/WAKU2-DISCV5 31/WAKU2-ENR multiaddress libp2p discovery interface libp2p gossipsub 29/WAKU2-CONFIG Waku relay anonymity `}),e.add({id:29,href:"/spec/35/",title:"35/WAKU2-NOISE",section:"Docs",content:`This specification describes how payloads of Waku messages with version 2 can be encrypted in order to achieve confidentiality, authenticity, and integrity as well as some form of identity-hiding on communicating parties.
This specification extends the functionalities provided by 26/WAKU-PAYLOAD, adding support to modern symmetric encryption primitives and asymmetric key-exchange protocols.
Specifically, it adds support to the ChaChaPoly cipher for symmetric authenticated encryption. It further describes how the Noise Protocol Framework can be used to exchange cryptographic keys and encrypt/decrypt messages in a way that the latter are authenticated and protected by strong forward secrecy.
This ultimately allows Waku applications to instantiate end-to-end encrypted communication channels with strong conversational security guarantees, as similarly done by 5/SECURE-TRANSPORT but in a more modular way, adapting key-exchange protocols to the knowledge communicating parties have of each other.
Design requirements # Confidentiality: the adversary should not be able to learn what data is being sent from one Waku endpoint to one or several other Waku endpoints. Strong forward secrecy: an active adversary cannot decrypt messages nor infer any information on the employed encryption key, even in the case he has access to communicating parties\u0026rsquo; long-term private keys (during or after their communication). Authenticity: the adversary should not be able to cause a Waku endpoint to accept messages coming from an endpoint different than their original senders. Integrity: the adversary should not be able to cause a Waku endpoint to accept data that has been tampered with. Identity-hiding: once a secure communication channel is established, a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient. Supported Cryptographic Protocols # Noise Protocols # Two parties executing a Noise protocol exchange one or more handshake messages and/or transport messages. A Noise protocol consists of one or more Noise handshakes. During a Noise handshake, two parties exchange multiple handshake messages. A handshake message contains ephemeral keys and/or static keys from one of the parties and an encrypted or unencrypted payload that can be used to transmit optional data. These public keys are used to perform a protocol-dependent sequence of Diffie-Hellman operations, whose results are all hashed into a shared secret key. After a handshake is complete, each party will then use the derived shared secret key to send and receive authenticated encrypted transport messages. We refer to Noise protocol framework specifications for the full details on how parties shared secret key is derived from each exchanged message.
Four Noise handshakes are currently supported: K1K1, XK1, XX, XXpsk0. Their description can be found in Appendix: Supported Handshakes Description. These are instantiated combining the following cryptographic primitives:
Curve25519 for Diffie-Hellman key-exchanges (32 bytes curve coordinates); ChaChaPoly for symmetric authenticated encryption (16 bytes authentication tag); SHA256 hash function used in HMAC and HKDF keys derivation chains (32 bytes output size); Content Topics and Message Nametags of Noise Handshake Messages # We note that all design requirements on exchanged messages would be satisfied only after a supported Noise handshake is completed, corresponding to a total of 1 Round Trip Time communication (1-RTT).
In particular, identity-hiding properties can be guaranteed only if the recommendation described in After-handshake are implemented.
In the following, we assume that communicating parties reciprocally know an initial contentTopic where they can send/receive the first handshake message(s). We further assume that messages sent over a certain contentTopic can be efficiently identified by their intended recipients thanks to an arbitrary 16 bytes long message-nametag field embedded in the message payload which is known in advance before messages reception.
The second handshake message MAY be sent/received with a message-nametag deterministically derived from the handshake state obtained after processing the first handshake message (using, for example, HKDF over the handshake hash value h). This allows
the recipient to efficiently continue the handshakes started by each initiator; the initiators to efficiently associate the recipient\u0026rsquo;s second handshake message to their first handshake message, However, this does not provide any identity-hiding guarantee to the recipient. After the second handshake message is correctly received by initiators, the recommendation described in After-handshake SHOULD be implemented to provide full identity-hiding guarantees for both initiator and recipient against passive attackers.
Encryption Primitives # The symmetric primitives supported are:
ChaChaPoly for authenticated encryption (16 bytes authentication tag). Specification # When 14/WAKU-MESSAGE version is set to 2, the corresponding WakuMessage\u0026rsquo;s payload will encapsulate the two fields handshake-message and transport-message.
The handshake-message field MAY contain
a Noise handhshake message (only encrypted/unencrypted public keys). The transport-message field MAY contain
a Noise handshake message payload (encrypted/unencrypted); a Noise transport message; a ChaChaPoly ciphertext. When a transport-message encodes a ChaChaPoly ciphertext, the corresponding handshake-message field MUST be empty.
The following fields are concatenated to form the payload field:
message-nametag: an arbitrary identifier for the Waku message (16 byte). If the underlying encryption primitive supports it, the contents of this field SHOULD be passed as additional data to the encryption and decryption routines. protocol-id: identifies the protocol or primitive in use (1 byte). Supported values are: 0: protocol specification omitted (set for after-handshake messages); 10: Noise protocol Noise_K1K1_25519_ChaChaPoly_SHA256; 11: Noise protocol Noise_XK1_25519_ChaChaPoly_SHA256; 12: Noise protocol Noise_XX_25519_ChaChaPoly_SHA256; 13: Noise protocol Noise_XXpsk0_25519_ChaChaPoly_SHA256; 30: ChaChaPoly symmetric encryption. handshake-message-len: the length in bytes of the Noise handshake message (1 byte). If protocol-id is not equal to 0, 10, 11, 12, 13, this field MUST be set to 0; handshake-message: the Noise handshake message (handshake-message-len bytes). If handshake-message-len is not 0, it contains the concatenation of one or more Noise Diffie-Hellman ephemeral or static keys encoded as in Public Keys Encoding; transport-message-len: the length in bytes of transport-message (8 bytes, stored in Little-Endian); transport-message: the transport message (transport-message-len bytes); Only during a Noise handshake, this field would contain the Noise handshake message payload. The symmetric encryption authentication data for transport-message, when present, is appended at the end of transport-message (16 bytes). ABNF # Using Augmented Backus-Naur form (ABNF) we have the following format:
; message nametag message-nametag = 16OCTET ; protocol ID protocol-id = 1OCTET ; contains the size of handshake-message handshake-message-len = 1OCTET ; contains one or more Diffie-Hellman public keys handshake-message = *OCTET ; contains the size of transport-message transport-message-len = *OCTET ; contains the transport message, eventually encrypted. ; If encrypted, authentication data is appended transport-message = *OCTET ; the Waku WakuMessage payload field payload = message-nametag protocol-id handshake-message-len handshake-message transport-message-len transport-message Protocol Payload Format # Based on the specified protocol-id, the Waku message payload field will encode different types of protocol-dependent messages.
In particular, if protocol-id is
0: payload encodes an after-handshake message. handshake-message-len MAY be 0; transport-message contains the Noise transport message; 10,11,12,13: payload encodes a supported Noise handshake message. transport-message contains the Noise transport message; 30: payload encapsulate a ChaChaPoly ciphertext ct. handshake-message-len is set to 0; transport-message contains the concatenation of the encryption nonce (12 bytes) followed by the ciphertext ct and the authentication data for ct (16 bytes); transport-message-len is set accordingly to transport-message length; Public Keys Serialization # Diffie-Hellman public keys can be trasmitted in clear or in encrypted form (cf. WriteMessage) with authentication data attached. To distinguish between these two cases, public keys are serialized as the concatenation of the following three fields:
flag: is equal to 1 if the public key is encrypted; 0 otherwise (1 byte); pk: if flag = 0, it contains an encoding of the X coordinate of the public key. If flag = 1, it contains a symmetric encryption of an encoding of the X coordinate of the public key, followed by encryption\u0026rsquo;s authentication data; The corresponding serialization is obtained as flag pk.
As regards the underlying supported cryptographic primitives:
Curve25519 public keys X coordinates are encoded in little-endian as 32 bytes arrays; ChaChaPoly authentication data consists of 16 bytes (nonces are implicitely defined by Noise processing rules). In all supported Noise protocols, parties\u0026rsquo; static public keys are transmitted encrypted (cf. EncryptAndHash), while ephemeral keys MAY be encrypted after a handshake is complete.
Padding # To prevent some metadata leakage, encrypted transport messages SHOULD be padded before encryption.
It is therefore recommended to right pad transport messages using RFC2630 so that their final length is a multiple of 248 bytes.
After-handshake # During the initial 1-RTT communication, handshake messages might be linked, depending on the message-nametag derivation rule implemented, to the respective parties through the contentTopic and message-nametag fields employed for such communication.
After a handshake is completed, parties MAY derive from their shared secret key (preferably using HKDF) two random nametag-secret-outbound and nametag-secret-inbound values used to deterministically derive two arbitrary-long ordered lists of message-nametag used to indentify outbound and inbound messages, respectively (e.g. the n-th inbound message-nametag MAY be computed as HKDF(nametag-secret-inbound || n)). This allows communicating parties to efficiently identify messages addressed to them sent over a certain contentTopic and thus minimize the number of trial decryptions.
When communicating, parties SHOULD set protocol-id to 0 to reduce metadata leakages and indicate that the message is an after-handshake message.
Each party SHOULD attach an (unencrypted) ephemeral key in handshake-message to every message sent. According to Noise processing rules, this allows updates to the shared secret key by hashing the result of an ephemeral-ephemeral Diffie-Hellman exchange every 1-RTT communication.
Backward Support for Symmetric/Asymmetric Encryption # It is possible to have backward compatibility to symmetric/asymmetric encryption primitives from 26/WAKU-PAYLOAD, effectively encapsulating payload encryption 14/WAKU-MESSAGE version 1 in version 2.
It suffices to extend the list of supported protocol-id to:
254: AES-256-GCM symmetric encryption; 255: ECIES asymmetric encryption. and set the transport-message field to the 26/WAKU-PAYLOAD data field, whenever these protocol-id values are set.
Namely, if protocol-id = 254, 255 then:
message-nametag: is empty; handshake-message-len: is set to 0; handshake-message: is empty; transport-message: contains the 26/WAKU-PAYLOAD data field (AES-256-GCM or ECIES, depending on protocol-id); transport-message-len is set accordingly to transport-message length; When a transport-message corresponding to protocol-id = 254, 255 is retrieved, it SHOULD be decoded as the data field in 26/WAKU-PAYLOAD specification.
Appendix: Supported Handshakes Description # Supported Noise handshakes address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:
Alice and Bob know each others\u0026rsquo; static key. Alice knows Bob\u0026rsquo;s static key; Alice and Bob share no key material and they don\u0026rsquo;t know each others\u0026rsquo; static key. Alice and Bob share some key material, but they don\u0026rsquo;t know each others\u0026rsquo; static key. Adversarial Model: an active attacker who compromised one party\u0026rsquo;s static key may lower the identity-hiding security guarantees provided by some handshakes. In our security model we exclude such adversary, but for completeness we report a summary of possible de-anonymization attacks that can be performed by an active attacker.
The K1K1 Handshake # If Alice and Bob know each others\u0026rsquo; static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a K1K1 handshake. Using Noise notation (Alice is on the left) this can be sketched as:
K1K1: -\u0026gt; s \u0026lt;- s ... -\u0026gt; e \u0026lt;- e, ee, es -\u0026gt; se We note that here only ephemeral keys are exchanged. This handshake is useful in case Alice needs to instantiate a new separate encrypted communication channel with Bob, e.g. opening multiple parallel connections, file transfers, etc.
Security considerations on identity-hiding (active attacker): no static key is transmitted, but an active attacker impersonating Alice can check candidates for Bob\u0026rsquo;s static key.
The XK1 Handshake # Here, Alice knows how to initiate a communication with Bob and she knows his public static key: such discovery can be achieved, for example, through a publicly accessible register of users\u0026rsquo; static keys, smart contracts, or through a previous public/private advertisement of Bob\u0026rsquo;s static key.
A Noise handshake pattern that suits this scenario is XK1:
XK1: \u0026lt;- s ... -\u0026gt; e \u0026lt;- e, ee, es -\u0026gt; s, se Within this handshake, Alice and Bob reciprocally authenticate their static keys s using ephemeral keys e. We note that while Bob\u0026rsquo;s static key is assumed to be known to Alice (and hence is not transmitted), Alice\u0026rsquo;s static key is sent to Bob encrypted with a key derived from both parties ephemeral keys and Bob\u0026rsquo;s static key.
Security considerations on identity-hiding (active attacker): Alice\u0026rsquo;s static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob\u0026rsquo;s static key against recorded/accepted exchanged handshake messages.
The XX and XXpsk0 Handshakes # If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an XX handshake, where each party tranXmits to the other its own static key.
The handshake goes as follows:
XX: -\u0026gt; e \u0026lt;- e, ee, s, es -\u0026gt; s, se We note that the main difference with XK1 is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange.
This handshake can be slightly changed in case both Alice and Bob pre-shares some secret psk which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called XXpsk0, goes as follow:
XXpsk0: -\u0026gt; psk, e \u0026lt;- e, ee, s, es -\u0026gt; s, se The main difference with XX is that Alice\u0026rsquo;s and Bob\u0026rsquo;s static keys, when transmitted, would be encrypted with a key derived from psk as well.
Security considerations on identity-hiding (active attacker): Alice\u0026rsquo;s static key is encrypted with forward secrecy to an authenticated party for both XX and XXpsk0 handshakes. In XX, Bob\u0026rsquo;s static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In XXpsk0, instead, Bob\u0026rsquo;s secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret psk but not through any static key), provided that psk was not previously compromised (in such case identity-hiding properties provided by the XX handshake applies).
References # 5/SECURE-TRANSPORT 10/WAKU2 26/WAKU-PAYLOAD 14/WAKU-MESSAGE Noise protocol Noise handshakes as key-exchange mechanism for Waku2 Augmented Backus-Naur form (ABNF) RFC2630 - Content-encryption Process and padding Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:30,href:"/spec/36/",title:"36/WAKU2-BINDINGS-API",section:"Docs",content:` Introduction # Native applications that wish to integrate Waku may not be able to use nwaku and its JSON RPC API due to constraints on packaging, performance or executables.
An alternative is to link existing Waku implementation as a static or dynamic library in their application.
This specification describes the C API that SHOULD be implemented by native Waku library and that SHOULD be used to consume them.
Design requirements # The API should be generic enough, so:
it can be implemented by both nwaku and go-waku C-Bindings, it can be consumed from a variety of languages such as C#, Kotlin, Swift, Rust, C++, etc. The selected format to pass data to and from the API is JSON.
It has been selected due to its widespread usage and easiness of use. Other alternatives MAY replace it in the future (C structure, protobuf) if it brings limitations that need to be lifted.
The API # General # JsonResponse type # All the API functions return a JsonResponse unless specified otherwise. The returned JsonResponse is a char * whose format depends on whether the function was executed successfully or not. And it is important to notice that this returned JsonReponse is returned from the latest argument, instead of from the actual function\u0026rsquo;s returned value.
On failure:
{ error: string; } For example:
{ \u0026#34;error\u0026#34;: \u0026#34;the error message\u0026#34; } On success:
{ result: any } The type of the result object depends on the function it was returned by.
JsonMessage type # A Waku Message in JSON Format:
{ payload: string; contentTopic: string; version: number; timestamp: number; } Fields:
payload: base64 encoded payload, waku_utils_base64_encode can be used for this. contentTopic: The content topic to be set on the message. version: The Waku Message version number. timestamp: Unix timestamp in nanoseconds. DecodedPayload type # A payload once decoded, used when a received Waku Message is encrypted:
interface DecodedPayload { pubkey?: string; signature?: string; data: string; padding: string; } Fields:
pubkey: Public key that signed the message (optional), hex encoded with 0x prefix, signature: Message signature (optional), hex encoded with 0x prefix, data: Decrypted message payload base64 encoded, padding: Padding base64 encoded. FilterSubscription type # The criteria to create subscription to a light node in JSON Format:
{ contentFilters: ContentFilter[]; pubsubTopic: string?; } Fields:
contentFilters: Array of ContentFilter being subscribed to / unsubscribed from. topic: Optional pubsub topic. ContentFilter type # { contentTopic: string; } Fields:
contentTopic: The content topic of a Waku message. StoreQuery type # Criteria used to retrieve historical messages
interface StoreQuery { pubsubTopic?: string; contentFilters?: ContentFilter[]; startTime?: number; endTime?: number; pagingOptions?: PagingOptions } Fields:
pubsubTopic: The pubsub topic on which messages are published. contentFilters: Array of ContentFilter to query for historical messages, startTime: The inclusive lower bound on the timestamp of queried messages. This field holds the Unix epoch time in nanoseconds. endTime: The inclusive upper bound on the timestamp of queried messages. This field holds the Unix epoch time in nanoseconds. pagingOptions: Paging information in PagingOptions format. StoreResponse type # The response received after doing a query to a store node:
interface StoreResponse { messages: JsonMessage[]; pagingOptions?: PagingOptions; } Fields:
messages: Array of retrieved historical messages in JsonMessage format. pagingOption: Paging information in PagingOptions format from which to resume further historical queries PagingOptions type # interface PagingOptions { pageSize: number; cursor?: Index; forward: bool; } Fields:
pageSize: Number of messages to retrieve per page. cursor: Message Index from which to perform pagination. If not included and forward is set to true, paging will be performed from the beginning of the list. If not included and forward is set to false, paging will be performed from the end of the list. forward: true if paging forward, false if paging backward Index type # interface Index { digest: string; receiverTime: number; senderTime: number; pubsubTopic: string; } Fields:
digest: Hash of the message at this Index. receiverTime: UNIX timestamp in nanoseconds at which the message at this Index was received. senderTime: UNIX timestamp in nanoseconds at which the message is generated by its sender. pubsubTopic: The pubsub topic of the message at this Index. Events # Asynchronous events require a callback to be registered. An example of an asynchronous event that might be emitted is receiving a message. When an event is emitted, this callback will be triggered receiving a JSON string of type JsonSignal.
JsonSignal type # { type: string; event: any; } Fields:
type: Type of signal being emitted. Currently, only message is available. event: Format depends on the type of signal. For example:
{ \u0026#34;type\u0026#34;: \u0026#34;message\u0026#34;, \u0026#34;event\u0026#34;: { \u0026#34;subscriptionId\u0026#34;: 1, \u0026#34;pubsubTopic\u0026#34;: \u0026#34;/waku/2/default-waku/proto\u0026#34;, \u0026#34;messageId\u0026#34;: \u0026#34;0x6496491e40dbe0b6c3a2198c2426b16301688a2daebc4f57ad7706115eac3ad1\u0026#34;, \u0026#34;wakuMessage\u0026#34;: { \u0026#34;payload\u0026#34;: \u0026#34;TODO\u0026#34;, \u0026#34;contentTopic\u0026#34;: \u0026#34;/my-app/1/notification/proto\u0026#34;, \u0026#34;version\u0026#34;: 1, \u0026#34;timestamp\u0026#34;: 1647826358000000000 } } } type event Type message JsonMessageEvent JsonMessageEvent type # Type of event field for a message event:
{ pubsubTopic: string; messageId: string; wakuMessage: JsonMessage; } pubsubTopic: The pubsub topic on which the message was received. messageId: The message id. wakuMessage: The message in JsonMessage format. extern void waku_set_event_callback(void* cb) # Register callback to act as event handler and receive application signals, which are used to react to asynchronous events in Waku.
Parameters
void* cb: callback that will be executed when an async event is emitted. The function signature for the callback should be void myCallback(char* jsonSignal) Node management # JsonConfig type # Type holding a node configuration:
interface JsonConfig { host?: string; port?: number; advertiseAddr?: string; nodeKey?: string; keepAliveInterval?: number; relay?: boolean; relayTopics?: Array\u0026lt;string\u0026gt;; gossipsubParameters?: GossipSubParameters; minPeersToPublish?: number filter?: boolean; discV5?: boolean; discV5BootstrapNodes?: Array\u0026lt;string\u0026gt;; discV5UDPPort?: number; store?: boolean; databaseURL?: string; storeRetentionMaxMessages?: number; storeRetentionTimeSeconds?: number; } Fields:
All fields are optional. If a key is undefined, or null, a default value will be set.
host: Listening IP address. Default 0.0.0.0. port: Libp2p TCP listening port. Default 60000. Use 0 for random. advertiseAddr: External address to advertise to other nodes. Can be ip4, ip6 or dns4, dns6. If null, the multiaddress(es) generated from the ip and port specified in the config (or default ones) will be used. Default: null. nodeKey: Secp256k1 private key in Hex format (0x123...abc). Default random. keepAliveInterval: Interval in seconds for pinging peers to keep the connection alive. Default 20. relay: Enable relay protocol. Default true. relayTopics: Array of pubsub topics that WakuRelay will automatically subscribe to when the node starts Default [] gossipSubParameters: custom gossipsub parameters. See GossipSubParameters section for defaults minPeersToPublish: The minimum number of peers required on a topic to allow broadcasting a message. Default 0. filter: Enable filter protocol. Default false. discV5: Enable DiscoveryV5. Default false discV5BootstrapNodes: Array of bootstrap nodes ENR discV5UDPPort: UDP port for DiscoveryV5 Default 9000 store: Enable store protocol to persist message history Default false databaseURL: url connection string. Accepts SQLite and PostgreSQL connection strings Default: sqlite3://store.db storeRetentionMaxMessages: max number of messages to store in the database. Default 10000 storeRetentionTimeSeconds: max number of seconds that a message will be persisted in the database. Default 2592000 (30d) For example: { \u0026#34;host\u0026#34;: \u0026#34;0.0.0.0\u0026#34;, \u0026#34;port\u0026#34;: 60000, \u0026#34;advertiseAddr\u0026#34;: \u0026#34;1.2.3.4\u0026#34;, \u0026#34;nodeKey\u0026#34;: \u0026#34;0x123...567\u0026#34;, \u0026#34;keepAliveInterval\u0026#34;: 20, \u0026#34;relay\u0026#34;: true, \u0026#34;minPeersToPublish\u0026#34;: 0 } GossipsubParameters type # Type holding custom gossipsub configuration:
interface GossipSubParameters { D?: number; D_low?: number; D_high?: number; D_score?: number; D_out?: number; HistoryLength?: number; HistoryGossip?: number; D_lazy?: number; GossipFactor?: number; GossipRetransmission?: number; HeartbeatInitialDelayMs?: number; HeartbeatIntervalSeconds?: number; SlowHeartbeatWarning?: number; FanoutTTLSeconds?: number; PrunePeers?: number; PruneBackoffSeconds?: number; UnsubscribeBackoffSeconds?: number; Connectors?: number; MaxPendingConnections?: number; ConnectionTimeoutSeconds?: number; DirectConnectTicks?: number; DirectConnectInitialDelaySeconds?: number; OpportunisticGraftTicks?: number; OpportunisticGraftPeers?: number; GraftFloodThresholdSeconds?: number; MaxIHaveLength?: number; MaxIHaveMessages?: number; IWantFollowupTimeSeconds?: number; } Fields:
All fields are optional. If a key is undefined, or null, a default value will be set.
d: optimal degree for a GossipSub topic mesh. Default 6 dLow: lower bound on the number of peers we keep in a GossipSub topic mesh Default 5 dHigh: upper bound on the number of peers we keep in a GossipSub topic mesh. Default 12 dScore: affects how peers are selected when pruning a mesh due to over subscription. Default 4 dOut: sets the quota for the number of outbound connections to maintain in a topic mesh. Default 2 historyLength: controls the size of the message cache used for gossip. Default 5 historyGossip: controls how many cached message ids we will advertise in IHAVE gossip messages. Default 3 dLazy: affects how many peers we will emit gossip to at each heartbeat. Default 6 gossipFactor: affects how many peers we will emit gossip to at each heartbeat. Default 0.25 gossipRetransmission: controls how many times we will allow a peer to request the same message id through IWANT gossip before we start ignoring them. Default 3 heartbeatInitialDelayMs: short delay in milliseconds before the heartbeat timer begins after the router is initialized. Default 100 milliseconds heartbeatIntervalSeconds: controls the time between heartbeats. Default 1 second slowHeartbeatWarning: duration threshold for heartbeat processing before emitting a warning. Default 0.1 fanoutTTLSeconds: controls how long we keep track of the fanout state. Default 60 seconds prunePeers: controls the number of peers to include in prune Peer eXchange. Default 16 pruneBackoffSeconds: controls the backoff time for pruned peers. Default 60 seconds unsubscribeBackoffSeconds: controls the backoff time to use when unsuscribing from a topic. Default 10 seconds connectors: number of active connection attempts for peers obtained through PX. Default 8 maxPendingConnections: maximum number of pending connections for peers attempted through px. Default 128 connectionTimeoutSeconds: timeout in seconds for connection attempts. Default 30 seconds directConnectTicks: the number of heartbeat ticks for attempting to reconnect direct peers that are not currently connected. Default 300 directConnectInitialDelaySeconds: initial delay before opening connections to direct peers. Default 1 second opportunisticGraftTicks: number of heartbeat ticks for attempting to improve the mesh with opportunistic grafting. Default 60 opportunisticGraftPeers: the number of peers to opportunistically graft. Default 2 graftFloodThresholdSeconds: If a GRAFT comes before GraftFloodThresholdSeconds has elapsed since the last PRUNE, then there is an extra score penalty applied to the peer through P7. Default 10 seconds maxIHaveLength: max number of messages to include in an IHAVE message, also controls the max number of IHAVE ids we will accept and request with IWANT from a peer within a heartbeat. Default 5000 maxIHaveMessages: max number of IHAVE messages to accept from a peer within a heartbeat. Default 10 iWantFollowupTimeSeconds: Time to wait for a message requested through IWANT following an IHAVE advertisement. Default 3 seconds seenMessagesTTLSeconds: configures when a previously seen message ID can be forgotten about. Default 120 seconds extern unsigned int waku_new(char* jsonConfig, char* jsonResp) # Instantiates a Waku node.
Parameters
[input] char* jsonConfig: JSON string containing the options used to initialize a waku node. Type JsonConfig. It can be NULL to use defaults.
[output] char* jsonResp: JsonResponse. Provides information indicating whether the operation succeded or not. In case or error, it tries to give as much detail as possible.
Ex1:
{ \u0026#34;result\u0026#34;: true } Ex2:
{ \u0026#34;error\u0026#34;: \u0026#34;The node key is missing.\u0026#34; } Returns
Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. extern unsigned int waku_start(char* jsonResp) # Starts a Waku node mounting all the protocols that were enabled during the Waku node instantiation.
Parameters
[output] char* jsonResp: JsonResponse. Provides information indicating whether the operation succeded or not. In case or error, it tries to give as much detail as possible.
Ex:
{ \u0026#34;result\u0026#34;: true } Returns
Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. extern unsigned int waku_stop(char* jsonResp) # Stops a Waku node.
Parameters
[output] char* jsonResp: JsonResponse. Provides information indicating whether the operation succeded or not. In case or error, it tries to give as much detail as possible.
Ex:
{ \u0026#34;result\u0026#34;: true } Returns
Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. extern char* waku_peerid() # Get the peer ID of the waku node.
Returns
A JsonResponse. If the execution is successful, the result field contains the peer ID as a string (base58 encoded).
For example:
{ \u0026#34;result\u0026#34;: \u0026#34;QmWjHKUrXDHPCwoWXpUZ77E8o6UbAoTTZwf1AD1tDC4KNP\u0026#34; } extern char* waku_listen_addresses() # Get the multiaddresses the Waku node is listening to.
Returns
A JsonResponse. If the execution is successful, the result field contains an array of multiaddresses. The multiaddresses are strings.
For example:
{ \u0026#34;result\u0026#34;: [ \u0026#34;/ip4/127.0.0.1/tcp/30303\u0026#34;, \u0026#34;/ip4/1.2.3.4/tcp/30303\u0026#34;, \u0026#34;/dns4/waku.node.example/tcp/8000/wss\u0026#34; ] } Connecting to peers # extern char* waku_add_peer(char* address, char* protocolId) # Add a node multiaddress and protocol to the waku node\u0026rsquo;s peerstore.
Parameters
char* address: A multiaddress (with peer id) to reach the peer being added. char* protocolId: A protocol we expect the peer to support. Returns
A JsonResponse. If the execution is successful, the result field contains the peer ID as a base58 string of the peer that was added.
For example:
{ \u0026#34;result\u0026#34;: \u0026#34;QmWjHKUrXDHPCwoWXpUZ77E8o6UbAoTTZwf1AD1tDC4KNP\u0026#34; } extern unsigned int waku_connect(char* address, int timeoutMs, char* jsonResp) # Dial peer using a multiaddress.
Parameters
[input] char* address: A multiaddress to reach the peer being dialed.
[input] int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout.
[output] JsonResponse. If the execution is successful, the result field is set to true.
Ex1:
{ \u0026#34;result\u0026#34;: true } Ex2:
{ \u0026#34;error\u0026#34;: \u0026#34;Timeout expired.\u0026#34; } Returns
Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. extern char* waku_connect_peerid(char* peerId, int timeoutMs) # Dial peer using its peer ID.
Parameters
1char* peerID: Peer ID to dial. The peer must be already known. It must have been added before with waku_add_peer or previously dialed with waku_connect. 2. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout.
Returns
A JsonResponse. If the execution is successful, the result field is set to true.
For example:
{ \u0026#34;result\u0026#34;: true } extern char* waku_disconnect(char* peerId) # Disconnect a peer using its peerID
Parameters
char* peerID: Peer ID to disconnect. Returns
A JsonResponse. If the execution is successful, the result field is set to true.
For example:
{ \u0026#34;result\u0026#34;: true } extern char* waku_peer_count() # Get number of connected peers.
Returns
A JsonResponse. If the execution is successful, the result field contains an integer which represents the number of connected peers.
For example:
{ \u0026#34;result\u0026#34;: 0 } extern char* waku_peers() # Retrieve the list of peers known by the Waku node.
Returns
A JsonResponse containing a list of peers. The list of peers has this format:
{ \u0026#34;result\u0026#34;: [ { \u0026#34;peerID\u0026#34;: \u0026#34;16Uiu2HAmJb2e28qLXxT5kZxVUUoJt72EMzNGXB47RedcBafeDCBA\u0026#34;, \u0026#34;protocols\u0026#34;: [ \u0026#34;/ipfs/id/1.0.0\u0026#34;, \u0026#34;/vac/waku/relay/2.0.0\u0026#34;, \u0026#34;/ipfs/ping/1.0.0\u0026#34; ], \u0026#34;addrs\u0026#34;: [ \u0026#34;/ip4/1.2.3.4/tcp/30303\u0026#34; ], \u0026#34;connected\u0026#34;: true } ] } Waku Relay # extern void waku_content_topic(char* applicationName, unsigned int applicationVersion, char* contentTopicName, char* encoding, char* outContentTopic) # Create a content topic string according to RFC 23.
Parameters
[input] char* applicationName [input] unsigned int applicationVersion [input] char* contentTopicName [input] char* encoding: depending on the payload, use proto, rlp or rfc26 [output] char* outContentTopic. Gets populated with a content topic formatted according to RFC 23. /{application-name}/{version-of-the-application}/{content-topic-name}/{encoding} extern char* waku_pubsub_topic(char* name, char* encoding, char* outPubsubTopic) # Create a pubsub topic string according to RFC 23.
Parameters
[input] char* name [input] char* encoding: depending on the payload, use proto, rlp or rfc26 [output] char* outPubsubTopic. Gets populated with a pubsub topic formatted according to RFC 23. /waku/2/{topic-name}/{encoding} extern void waku_default_pubsub_topic(char* defaultPubsubTopic) # Returns the default pubsub topic used for exchanging waku messages defined in RFC 10.
Parameters
[output] char* defaultPubsubTopic. Gets populated with /waku/2/default-waku/proto extern unsinged int waku_relay_publish(char* messageJson, char* pubsubTopic, int timeoutMs, char* jsonResp) # Publish a message using Waku Relay.
Parameters
[input] char* messageJson: JSON string containing the Waku Message as JsonMessage. [input] char* pubsubTopic: pubsub topic on which to publish the message. If NULL, it uses the default pubsub topic. [input] int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. [output]: A JsonResponse. If the execution is successful, the result field contains the message ID. Note: messageJson.version is overwritten to 0.
Returns Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. extern char* waku_relay_publish_enc_asymmetric(char* messageJson, char* pubsubTopic, char* publicKey, char* optionalSigningKey, int timeoutMs) # Optionally sign, encrypt using asymmetric encryption and publish a message using Waku Relay.
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* pubsubTopic: pubsub topic on which to publish the message. If NULL, it uses the default pubsub topic. char* publicKey: hex encoded public key to be used for encryption. char* optionalSigningKey: hex encoded private key to be used to sign the message. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Note: messageJson.version is overwritten to 1.
Returns
A JsonResponse. If the execution is successful, the result field contains the message ID.
extern char* waku_relay_publish_enc_symmetric(char* messageJson, char* pubsubTopic, char* symmetricKey, char* optionalSigningKey, int timeoutMs) # Optionally sign, encrypt using symmetric encryption and publish a message using Waku Relay.
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* pubsubTopic: pubsub topic on which to publish the message. If NULL, it uses the default pubsub topic. char* symmetricKey: hex encoded secret key to be used for encryption. char* optionalSigningKey: hex encoded private key to be used to sign the message. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Note: messageJson.version is overwritten to 1.
Returns
A JsonResponse. If the execution is successful, the result field contains the message ID.
extern char* waku_relay_enough_peers(char* pubsubTopic) # Determine if there are enough peers to publish a message on a given pubsub topic.
Parameters
char* pubsubTopic: Pubsub topic to verify. If NULL, it verifies the number of peers in the default pubsub topic. Returns
A JsonResponse. If the execution is successful, the result field contains a boolean indicating whether there are enough peers.
For example:
{ \u0026#34;result\u0026#34;: true } extern unsigned int waku_relay_subscribe(char* topic, char* jsonResp) # Subscribe to a Waku Relay pubsub topic to receive messages.
Parameters
[input] char* topic: Pubsub topic to subscribe to. If NULL, it subscribes to the default pubsub topic.
[output] char* jsonResp: A JsonResponse. If the execution is successful, the result field is set to true.
Ex1:
{ \u0026#34;result\u0026#34;: true } { \u0026#34;error\u0026#34;: \u0026#34;Cannot subscribe without Waku Relay enabled.\u0026#34; } Returns Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. Events
When a message is received, a \`\`\u0026ldquo;message\u0026rdquo; event is emitted containing the message, pubsub topic, and node ID in which the message was received.
The event type is JsonMessageEvent.
For Example:
{ \u0026#34;type\u0026#34;: \u0026#34;message\u0026#34;, \u0026#34;event\u0026#34;: { \u0026#34;pubsubTopic\u0026#34;: \u0026#34;/waku/2/default-waku/proto\u0026#34;, \u0026#34;messageId\u0026#34;: \u0026#34;0x6496491e40dbe0b6c3a2198c2426b16301688a2daebc4f57ad7706115eac3ad1\u0026#34;, \u0026#34;wakuMessage\u0026#34;: { \u0026#34;payload\u0026#34;: \u0026#34;TODO\u0026#34;, \u0026#34;contentTopic\u0026#34;: \u0026#34;/my-app/1/notification/proto\u0026#34;, \u0026#34;version\u0026#34;: 1, \u0026#34;timestamp\u0026#34;: 1647826358000000000 } } } extern unsigned int waku_relay_unsubscribe(char* topic, char* jsonResp) # Closes the pubsub subscription to a pubsub topic. No more messages will be received from this pubsub topic.
Parameters
[input] char* pusubTopic: Pubsub topic to unsubscribe from. If NULL, unsubscribes from the default pubsub topic.
[output] A JsonResponse. If the execution is successful, the result field is set to true.
For example:
{ \u0026#34;result\u0026#34;: true } Returns Unsigned int. Possible values:
1 - The operation was completed successfuly. 0 - The operation failed for any reason. It worth checking the value of jsonResp in this case. extern char* waku_relay_topics() # Get the list of subscribed pubsub topics in Waku Relay.
Returns
A JsonResponse. If the execution is successful, the result field will contain an array of pubsub topics.
For example:
{ \u0026#34;result\u0026#34;: [\u0026#34;pubsubTopic1\u0026#34;, \u0026#34;pubsubTopic2\u0026#34;] } Waku Filter # extern char* waku_filter_subscribe(char* filterJSON, char* peerID, int timeoutMs) # Creates a subscription in a lightnode for messages that matches a content filter and optionally a PubSub topic.
Parameters
char* filterJSON: JSON string containing the FilterSubscription to subscribe to. char* peerID: Peer ID to subscribe to. The peer must be already known. It must have been added before with waku_add_peer or previously dialed with waku_connect_peer. Use NULL to automatically select a node. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Returns
A JsonResponse. If the execution is successful, the result field is set to true.
For example:
{ \u0026#34;result\u0026#34;: true } Events
When a message is received, a \`\`\u0026ldquo;message\u0026rdquo; event is emitted containing the message, pubsub topic, and node ID in which the message was received.
The event type is JsonMessageEvent.
For Example:
{ \u0026#34;type\u0026#34;: \u0026#34;message\u0026#34;, \u0026#34;event\u0026#34;: { \u0026#34;pubsubTopic\u0026#34;: \u0026#34;/waku/2/default-waku/proto\u0026#34;, \u0026#34;messageId\u0026#34;: \u0026#34;0x6496491e40dbe0b6c3a2198c2426b16301688a2daebc4f57ad7706115eac3ad1\u0026#34;, \u0026#34;wakuMessage\u0026#34;: { \u0026#34;payload\u0026#34;: \u0026#34;TODO\u0026#34;, \u0026#34;contentTopic\u0026#34;: \u0026#34;/my-app/1/notification/proto\u0026#34;, \u0026#34;version\u0026#34;: 1, \u0026#34;timestamp\u0026#34;: 1647826358000000000 } } } extern char* waku_filter_unsubscribe(char* filterJSON, int timeoutMs) # Removes subscriptions in a light node matching a content filter and, optionally, a PubSub topic.
Parameters
char* filterJSON: JSON string containing the FilterSubscription. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Returns
A JsonResponse. If the execution is successful, the result field is set to true.
For example:
{ \u0026#34;result\u0026#34;: true } Waku Lightpush # extern char* waku_lightpush_publish(char* messageJSON, char* topic, char* peerID, int timeoutMs) # Publish a message using Waku Lightpush.
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* pubsubTopic: pubsub topic on which to publish the message. If NULL, it uses the default pubsub topic. char* peerID: Peer ID supporting the lightpush protocol. The peer must be already known. It must have been added before with waku_add_peer or previously dialed with waku_connect_peer. Use NULL to automatically select a node. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Note: messageJson.version is overwritten to 0.
Returns
A JsonResponse. If the execution is successful, the result field contains the message ID.
extern char* waku_lightpush_publish_enc_asymmetric(char* messageJson, char* pubsubTopic, char* peerID, char* publicKey, char* optionalSigningKey, int timeoutMs) # Optionally sign, encrypt using asymmetric encryption and publish a message using Waku Lightpush.
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* pubsubTopic: pubsub topic on which to publish the message. If NULL, it uses the default pubsub topic. char* peerID: Peer ID supporting the lightpush protocol. The peer must be already known. It must have been added before with waku_add_peer or previously dialed with waku_connect_peer. char* publicKey: hex encoded public key to be used for encryption. char* optionalSigningKey: hex encoded private key to be used to sign the message. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Note: messageJson.version is overwritten to 1.
Returns
A JsonResponse. If the execution is successful, the result field contains the message ID.
extern char* waku_lightpush_publish_enc_symmetric(char* messageJson, char* pubsubTopic, char* peerID, char* symmetricKey, char* optionalSigningKey, int timeoutMs) # Optionally sign, encrypt using symmetric encryption and publish a message using Waku Lightpush.
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* pubsubTopic: pubsub topic on which to publish the message. If NULL, it uses the default pubsub topic. char* peerID: Peer ID supporting the lightpush protocol. The peer must be already known. It must have been added before with waku_add_peer or previously dialed with waku_connect_peer. char* symmetricKey: hex encoded secret key to be used for encryption. char* optionalSigningKey: hex encoded private key to be used to sign the message. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Note: messageJson.version is overwritten to 1.
Returns
A JsonResponse. If the execution is successful, the result field contains the message ID.
Waku Store # extern char* waku_store_query(char* queryJSON, char* peerID, int timeoutMs) # Retrieves historical messages on specific content topics. This method may be called with PagingOptions, to retrieve historical messages on a per-page basis. If the request included PagingOptions, the node must return messages on a per-page basis and include PagingOptions in the response. These PagingOptions must contain a cursor pointing to the Index from which a new page can be requested.
Parameters
char* queryJSON: JSON string containing the StoreQuery. char* peerID: Peer ID supporting the store protocol. The peer must be already known. It must have been added before with waku_add_peer or previously dialed with waku_connect_peer. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Returns
A JsonResponse. If the execution is successful, the result field contains a StoreResponse..
Decrypting messages # extern char* waku_decode_symmetric(char* messageJson, char* symmetricKey) # Decrypt a message using a symmetric key
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* symmetricKey: 32 byte symmetric key hex encoded. Note: messageJson.version is expected to be 1.
Returns
A JsonResponse. If the execution is successful, the result field contains the decoded payload as a DecodedPayload. An error message otherwise.
{ \u0026#34;result\u0026#34;: { \u0026#34;pubkey\u0026#34;: \u0026#34;0x......\u0026#34;, \u0026#34;signature\u0026#34;: \u0026#34;0x....\u0026#34;, \u0026#34;data\u0026#34;: \u0026#34;...\u0026#34;, \u0026#34;padding\u0026#34;: \u0026#34;...\u0026#34; } } extern char* waku_decode_asymmetric(char* messageJson, char* privateKey) # Decrypt a message using a secp256k1 private key
Parameters
char* messageJson: JSON string containing the Waku Message as JsonMessage. char* privateKey: secp256k1 private key hex encoded. Note: messageJson.version is expected to be 1.
Returns
A JsonResponse. If the execution is successful, the result field contains the decoded payload as a DecodedPayload. An error message otherwise.
{ \u0026#34;result\u0026#34;: { \u0026#34;pubkey\u0026#34;: \u0026#34;0x......\u0026#34;, \u0026#34;signature\u0026#34;: \u0026#34;0x....\u0026#34;, \u0026#34;data\u0026#34;: \u0026#34;...\u0026#34;, \u0026#34;padding\u0026#34;: \u0026#34;...\u0026#34; } } DNS Discovery # extern char* waku_dns_discovery(char* url, char* nameserver, int timeoutMs) # Returns a list of multiaddress given a url to a DNS discoverable ENR tree
Parameters
char* url: URL containing a discoverable ENR tree char* nameserver: The nameserver to resolve the ENR tree url. If NULL or empty, it will automatically use the default system dns. int timeoutMs: Timeout value in milliseconds to execute the call. If the function execution takes longer than this value, the execution will be canceled and an error returned. Use 0 for no timeout. Returns
A JsonResponse. If the execution is successful, the result field contains an array objects describing the multiaddresses, enr and peerID each node found. An error message otherwise.
{ \u0026#34;result\u0026#34;:[ { \u0026#34;peerID\u0026#34;:\u0026#34;16Uiu2HAmPLe7Mzm8TsYUubgCAW1aJoeFScxrLj8ppHFivPo97bUZ\u0026#34;, \u0026#34;multiaddrs\u0026#34;:[ \u0026#34;/ip4/134.209.139.210/tcp/30303/p2p/16Uiu2HAmPLe7Mzm8TsYUubgCAW1aJoeFScxrLj8ppHFivPo97bUZ\u0026#34;, \u0026#34;/dns4/node-01.do-ams3.wakuv2.test.statusim.net/tcp/8000/wss/p2p/16Uiu2HAmPLe7Mzm8TsYUubgCAW1aJoeFScxrLj8ppHFivPo97bUZ\u0026#34; ], \u0026#34;enr\u0026#34;:\u0026#34;enr:-M-4QCtJKX2WDloRYDT4yjeMGKUCRRcMlsNiZP3cnPO0HZn6IdJ035RPCqsQ5NvTyjqHzKnTM6pc2LoKliV4CeV0WrgBgmlkgnY0gmlwhIbRi9KKbXVsdGlhZGRyc7EALzYobm9kZS0wMS5kby1hbXMzLndha3V2Mi50ZXN0LnN0YXR1c2ltLm5ldAYfQN4DiXNlY3AyNTZrMaEDnr03Tuo77930a7sYLikftxnuG3BbC3gCFhA4632ooDaDdGNwgnZfg3VkcIIjKIV3YWt1Mg8\u0026#34; }, ... } Utils # extern char* waku_utils_base64_encode(char* data) # Encode a byte array to base64. Useful for creating the payload of a Waku Message in the format understood by waku_relay_publish
Parameters
char* data: Byte array to encode Returns
A char * containing the base64 encoded byte array.
extern char* waku_utils_base64_decode(char* data) # Decode a base64 string (useful for reading the payload from Waku Messages).
Parameters
char* data: base64 encoded byte array to decode. Returns
A JsonResponse. If the execution is successful, the result field contains the decoded payload.
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:31,href:"/spec/37/",title:"37/WAKU2-NOISE-SESSIONS",section:"Docs",content:` Introduction # In 35/WAKU2-NOISE we defined how Waku messages\u0026rsquo; payloads can be encrypted using key material derived from key agreements based on the Noise Protocol Framework.
Once two users complete a Noise handshake, an encryption/decryption session - or a Noise session - would be instantiated.
This post provides an overview on how we can possibly implement and manage one or multiple Noise sessions in Waku.
Preliminaries # We assume that two users, e.g. Alice and Bob, successfully completed a Noise handshake.
Using Noise terminology, at the end of the handshake they will share:
two Cipher States CSOutbound and CSInbound, to encrypt and decrypt outbound and inbound messages, respectively; a handshake hash value h. As suggested in Noise specifications in regards to Channel Binding, we can identify a Noise session with a session-id derived from the handshake hash value h shared on completion of a Noise handshake.
More specifically, when Alice and Bob call Split() in order to derive the two final encryption and decryption Cipher States, they further compute session-id = HKDF(h) using the supported key derivation function HKDF.
Such session-id will uniquely identify the Noise cryptographic session instantiated on completion of a Noise handshake, which would then consist of the tuple (session-id, CSOutbound, CSInbound). For each instantiated Noise session we assume this tuple to be properly persisted, since it is required to either retrieve and encrypt/decrypt any further exchanged message.
Once a Noise session is instantiated, any further encrypted message between Alice and Bob within this session is exchanged on a contentTopic with name /{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto, where ct-id = Hash(Hash(session-id)) and /{application-name}/{application-version}/ identifies the application currently employing 35/WAKU2-NOISE.
Session states # A Noise session corresponding to a certain session-id:
is always active as long as it is not marked as stale. For an active session-id, new messages are published on the content topic /{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto; is marked as stale if a session termination message containing Hash(session-id) is published on the content topic /{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto. Session information relative to stale sessions MAY be deleted from users\u0026rsquo; device, unless required for later channel binding purposes. When a Noise session is marked as stale, it means that one party requested its termination while being online, since publication of a hash pre-image for ct-id is required (i.e. Hash(session-id)).
Currently, it is not possible to mark a Noise session as stale when session-id is lost or gets corrupted in users\u0026rsquo; devices. However, since session-id is shared between Alice and Bob, one party MAY decide to mark a Noise session as stale if no message from the other end was received within a certain fixed time window.
The above mechanism allows a Noise session to be marked as stale either privately or publicly, depending if Hash(session-id) is sent on /{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto to the other party in encrypted form or not, respectively.
When a Noise session is publicly marked as stale, network peers MAY discard all stored messages addressed to the content topic /{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto. In this the case and in order for parties to retrieve any eventually delayed message, peers SHOULD wait a fixed amount of time before discarding stored messages corresponding to a stale Noise session.
A stale Noise session cannot be directly marked as active and parties are required to instantiate a new Noise session if they wish to communicate again.
However, parties can optionally persist and include the session-id corresponding to a stale Noise session in the prologue information employed in the Noise handshake they execute to instantiate their new Noise session. This effectively emulates a mechanism to \u0026ldquo;re-activate\u0026rdquo; a stale Noise session by binding it to a newly created active Noise session.
In order to reduce users\u0026rsquo; metadata leakage, it is desirable (as suggested in 35/WAKU2-NOISE) that content topics used for communications change every time a new message is exchanged. This can be easily realized by employing a key derivation function to compute a new session-id from the previously employed one (e.g. session-id = HKDF(prev-session-id)), while keeping the Inbound/outbound Cipher States, the content topic derivation mechanism and the stale mechanism the same as above. In this case, when one party sends and receives at least one message, he SHALL publicly mark as stale all Noise sessions relative to messages exchanged before the earlier of these two send/receive events.
Multi-Device support # Alice and Bob might possess one or more devices (e.g. laptops, smartphones, etc.) they wish to use to communicate. In the following, we assume Alice and Bob to possess $N$ and $M$ devices, respectively.
Since a Noise session contains cryptographic material required to encrypt and decrypt messages exchanged on a pre-defined content topic derived from a session-id, messages should be encrypted and decrypted within the Noise session instantiated between the currently-in-use sender\u0026rsquo;s and receiver\u0026rsquo;s device.
This is achieved through two main supported session management mechanisms that we called N11M and NM, respectively.
The $N11M$ session management mechanism # In a $N11M$ setting, each party\u0026rsquo;s device shares the same Noise session information used to encrypt and decrypt messages exchanged with the other party.
More precisely, once the first Noise session between any of Alice\u0026rsquo;s and Bob\u0026rsquo;s device is instantiated, its session information is securely propagated to all other devices, which then become able to send and receive new messages on the content topic associated to such session. We note, however, that two devices belonging to one party cannot simultaneously send different messages to the other, since only the first message received will be correctly decrypted using the next nonce.
The most updated session information between Alice and Bob is propagated in encrypted form to other devices, using previously instantiated Noise sessions. In particular, all Alice\u0026rsquo;s (resp., Bob\u0026rsquo;s) devices that want to receive such updated session information, are required to have an already instantiated Noise session between them in order to receive it in encrypted form. The propagated session information corresponds to the latest session information stored on the device currently communicating with (any of the devices of) the other party.
We note that sessions information is propagated only among devices belonging to the same party and not with other party\u0026rsquo;s devices. Hence, Alice has no knowledge on the number of devices Bob is using and vice versa.
When any device marks a Noise session between Alice and Bob as stale, all other (updated) devices will consider such session as stale without publishing the Hash(session-id) on the corresponding session content topic.
In case a Noise session between two devices belonging to the same party is marked as stale, such two devices stop to reciprocally propagate any information regarding Noise sessions instantiated with other parties.
As regards security, an attacker that compromises an encrypted message propagating session information, might be able to compromise one or multiple messages exchanged within the session such information refers to. This can be mitigated by adopting techniques similar to the the ones proposed in 35/WAKU2-NOISE, where encryption keys are changed every time a new message is exchanged.
This session management mechanism is loosely based on the paper \u0026ldquo;Multi-Device for Signal\u0026rdquo;.
The $NM$ session management mechanism # In a $NM$ setting, we require all of $N$ Alice\u0026rsquo;s devices to have an active Noise session with each of Bob\u0026rsquo;s $M$ devices, for a total of $NM$ concurrently active Noise sessions between Alice and Bob.
A message is sent from the currently-in-use sender\u0026rsquo;s device to all recipent\u0026rsquo;s devices, by properly encrypting and sending it to the content topics of each corresponding active Noise session.
We note that this allows the recipient to receive a message on all his devices simultaneously. However, on the sender side, only the device which effectively sent the message will know its full content.
If it is required for sent messages to be available on all sender\u0026rsquo;s devices, each pair of sender\u0026rsquo;s devices SHOULD have an active Noise session used for syncing purposes: this sums up to a total of $N-1$ and $M-1$ extra Noise sessions instantiated on each Alice\u0026rsquo;s and Bob\u0026rsquo;s device, respectively.
Thus, if Alice wants to send a message to Bob from one of her $N$ devices, she encrypts and sends her message to each of Bob\u0026rsquo;s $M$ devices (and, eventually, to each of her other $N-1$ devices), using the appropriate Noise session information.
If one device marks a Noise session as stale, all active sessions instantiated with such device SHOULD be marked as stale as soon as possible. If the device declaring a stale session does not send a session termination message to all the other party\u0026rsquo;s devices with which has an active session, the other party SHOULD send a termination message to mark all such Noise sessions as stale.
This session management mechanism is loosely based on Signal\u0026rsquo;s Sesame Algorithm.
References # 13/WAKU2-STORE 35/WAKU2-NOISE The Noise Protocol Framework The Sesame Algorithm: Session Management for Asynchronous Message Encryption \u0026ldquo;Multi-Device for Signal\u0026rdquo; `}),e.add({id:32,href:"/spec/38/",title:"38/CONSENSUS-CLARO",section:"Docs",content:` Abstract # This document specifies Claro: a Byzantine, fault-tolerant, binary decision agreement algorithm that utilizes bounded memory for its execution. Claro is a novel variant of the Snow family providing a probabilistic leaderless BFT consensus algorithm that achieves metastablity via network sub-sampling. We present an application context of the use of Claro in an efficient, leaderless, probabilistic permission-less consensus mechanism. We outline a simple taxonomy of Byzantine adversaries, leaving explicit explorations of to subsequent publication.
NOTE: We have renamed this variant to Claro from Glacier in order to disambiguate from a previously released research endeavor by Amores-Sesar, Cachin, and Tedeschi. Their naming was coincidentally named the same as our work but is sufficiently differentiated from how ours works.
Motivation # This work is a part of a larger research endeavor to explore highly scalable Byzantine Fault Tolerant (BFT) consensus protocols. Consensus lies at the heart of many decentralized protocols, and thus its characteristics and properties are inherited by applications built on top. Thus, we seek to improve upon the current state of the art in two main directions: base-layer scalability and censorship resistance.
Avalanche has shown to exibit the former in a production environment in a way that is differentiated from Nakamoto consensus and other Proof of Stake (PoS) protocols based in practical Byzantine Fault Tolerant (pBFT) methodologies. We aim to understand its limitations and improve upon them.
Background # Our starting point is Avalanche’s Binary Byzantine Agreement algorithm, called Snowball. As long as modifications allow a DAG to be constructed later on, this simplifies the design significantly. The DAG stays the same in principle: it supports confidence, but the core algorithm can be modeled without.
The concept of the Snowball algorithm is relatively simple. Following is a simplified description (lacking some details, but giving an overview). For further details, please refer to the Avalanche paper.
The objective is to vote yes/no on a decision (this decision could be a single bit, or, in our DAG use case, whether a vertex should be included or not). Every node has an eventually-consistent complete view of the network. It will select at random k nodes, and will ask their opinion on the decision (yes/no). After this sampling is finished, if there is a vote that has more than an alpha threshold, it accumulates one count for this opinion, as well as changes its opinion to this one. But, if a different opinion is received, the counter is reset to 1. If no threshold alpha is reached, the counter is reset to 0 instead. After several iterations of this algorithm, we will reach a threshold beta, and decide on that as final. Next, we will proceed to describe our new algorithm, based on Snowball.
We have identified a shortcoming of the Snowball algorithm that was a perfect starting point for devising improvements. The scenario is as follows:
There is a powerful adversary in the network, that controls a large percentage of the node population: 10% to ~50%. This adversary follows a strategy that allows them to rapidly change the decision bit (possibly even in a coordinated way) so as to maximally confuse the honest nodes. Under normal conditions, honest nodes will accumulate supermajorities soon enough, and reach the beta threshold. However, when an honest node performs a query and does not reach the threshold alpha of responses, the counter will be set to 0. The highest threat to Snowball is an adversary that keeps it from reaching the beta threshold, managing to continuously reset the counter, and steering Snowball away from making a decision. This document only outlines the specification to Claro. Subsequent analysis work on Claro (both on its performance and how it differentiates with Snowball) will be published shortly and this document will be updated.
Claro Algorithm Specification # The Claro consensus algorithm computes a boolean decision on a proposition via a set of distributed computational nodes. Claro is a leaderless, probabilistic, binary consensus algorithm with fast finality that provides good reliability for network and Byzantine fault tolerance.
Algorithmic concept # Claro is an evolution of the Snowball Byzantine Binary Agreement (BBA) algorithm, in which we tackle specifically the perceived weakness described above. The main focus is going to be the counter and the triggering of the reset. Following, we elaborate the different modifications and features that have been added to the reference algorithm:
Instead of allowing the latest evidence to change the opinion completely, we take into account all accumulated evidence, to reduce the impact of high variability when there is already a large amount of evidence collected. Eliminate the counter and threshold scheme, and introduce instead two regimes of operation: One focused on grabbing opinions and reacting as soon as possible. This part is somewhat closer conceptually to the reference algorithm. Another one focused on interpreting the accumulated data instead of reacting to the latest information gathered. Finally, combine those two phases via a transition function. This avoids the creation of a step function, or a sudden change in behavior that could complicate analysis and understanding of the dynamics. Instead, we can have a single algorithm that transfers weight from one operation to the other as more evidence is gathered. Additionally, we introduce a function for weighted sampling. This will allow the combination of different forms of weighting: Staking Heuristic reputation Manual reputation. It’s worth delving a bit into the way the data is interpreted in order to reach a decision. Our approach is based conceptually on the paper Confidence as Higher-Order Uncertainty, which describes a frequentist approach to decision certainty. The first-order certainty, measured by frequency, is caused by known positive evidence, and the higher-order certainty is caused by potential positive evidence. Because confidence is a relative measurement defined on evidence, it naturally follows comparing the amount of evidence the system knows with the amount that it will know in the near future (defining “near” as a constant).
Intuitively, we are looking for a function of evidence, w, call it c for confidence, that satisfies the following conditions:
Confidence c is a continuous and monotonically increasing function of w. (More evidence, higher confidence.) When w = 0, c = 0. (Without any evidence, confidence is minimum.) When w goes to infinity, c converges to 1. (With infinite evidence, confidence is maximum.) The paper describes also a set of operations for the evidence/confidence pairs, so that different sources of knowledge could be combined. However, we leave here the suggestion of a possible research line in the future combining an algebra of evidence/confidence pairs with swarm-propagation algorithm like the one described in this paper.
Initial opinion # A proposal is formulated to which consensus of truth or falsity is desired. Each node that participates starts the protocol with an opinion on the proposal, represented in the sequel as NO, NONE, and YES.
A new proposition is discovered either by local creation or in response to a query, a node checks its local opinion. If the node can compute a justification of the proposal, it sets its opinion to one of YES or NO. If it cannot form an opinion, it leaves its opinion as NONE.
For now, we will ignore the proposal dissemination process and assume all nodes participating have an initial opinion to respond to within a given request. Further research will relax this assumption and analyze timing attacks on proposal propagation through the network.
The node then participates in a number of query rounds in which it solicits other node\u0026rsquo;s opinion in query rounds. Given a set of N leaderless computational nodes, a gossip-based protocol is presumed to exist which allows members to discover, join, and leave a weakly transitory maximally connected graph. Joining this graph allows each node to view a possibly incomplete node membership list of all other nodes. This view may change as the protocol advances, as nodes join and leave. Under generalized Internet conditions, the membership of the graph would experience a churn rate varying across different time-scales, as the protocol rounds progress. As such, a given node may not have a view on the complete members participating in the consensus on a proposal in a given round.
The algorithm is divided into 4 phases:
Querying Computing confidence, evidence, and accumulated evidence Transition function Opinion and Decision Setup Parameters # The node initializes the following integer ratios as constants:
# The following values are constants chosen with justification from experiments # performed with the adversarial models # confidence_threshold \u0026lt;-- 1 # constant look ahead for number of rounds we expect to finalize a # decision. Could be set dependent on number of nodes # visible in the current gossip graph. look_ahead \u0026lt;-- 19 # the confidence weighting parameter (aka alpha_1) certainty \u0026lt;-- 4 / 5 doubt ;; the lack of confidence weighting parameter (aka alpha_2) \u0026lt;-- 2 / 5 k_multiplier ;; neighbor threshold multiplier \u0026lt;-- 2 ;;; maximal threshold multiplier, i.e. we will never exceed ;;; questioning k_initial * k_multiplier ^ max_k_multiplier_power peers max_k_multiplier_power \u0026lt;-- 4 ;;; Initial number of nodes queried in a round k_initial \u0026lt;-- 7 ;;; maximum query rounds before termination max_rounds ;; placeholder for simulation work, no justification yet \u0026lt;-- 100 The following variables are needed to keep the state of Claro:
;; current number of nodes to attempt to query in a round k \u0026lt;-- k_original ;; total number of votes examined over all rounds total_votes \u0026lt;-- 0 ;; total number of YES (i.e. positive) votes for the truth of the proposal total_positive \u0026lt;-- 0 ;; the current query round, an integer starting from zero round \u0026lt;-- 0 Phase One: Query # A node selects k nodes randomly from the complete pool of peers in the network. This query is can optionally be weighted, so the probability of selecting nodes is proportional to their
Node Weighting $$ P(i) = \\frac{w_i}{\\sum_{j=0}^{j=N} w_j} $$
where w is evidence. The list of nodes is maintained by a separate protocol (the network layer), and eventual consistency of this knowledge in the network suffices. Even if there are slight divergences in the network view from different nodes, the algorithm is resilient to those.
A query is sent to each neighbor with the node\u0026rsquo;s current opinion of the proposal.
Each node replies with their current opinion on the proposal.
See the wire protocol Interoperability section for details on the semantics and syntax of the \u0026ldquo;on the wire\u0026rdquo; representation of this query.
Adaptive querying. An additional optimization in the query consists of adaptively growing the k constant in the event of high confusion. We define high confusion as the situation in which neither opinion is strongly held in a query (i.e. a threshold is not reached for either yes or no). For this, we will use the alpha threshold defined below. This adaptive growth of the query size is done as follows:
Every time the threshold is not reached, we multiply k by a constant. In our experiments, we found that a constant of 2 works well, but what really matters is that it stays within that order of magnitude.
The growth is capped at 4 times the initial k value. Again, this is an experimental value, and could potentially be increased. This depends mainly on complex factors such as the size of the query messages, which could saturate the node bandwidth if the number of nodes queried is too high.
When the query finishes, the node now initializes the following two values:
new_votes \u0026lt;-- |total vote replies received in this round to the current query| positive_votes \u0026lt;-- |YES votes received from the query| Phase Two: Computation # When the query returns, three ratios are used later on to compute the transition function and the opinion forming. Confidence encapsulates the notion of how much we know (as a node) in relation to how much we will know in the near future (this being encoded in the look-ahead parameter l.) Evidence accumulated keeps the ratio of total positive votes vs the total votes received (positive and negative), whereas the evidence per round stores the ratio of the current round only.
Parameters $$ \\begin{array}{lc} \\text{Look-ahead parameter} \u0026amp; l = 20 \\newline \\text{First evidence parameter} \u0026amp; \\alpha_1 = 0.8 \\newline \\text{Second evidence parameter} \u0026amp; \\alpha_2 = 0.5 \\newline \\end{array} $$
Computation $$ \\begin{array}{lc} \\text{Confidence} \u0026amp; c_{accum} \\impliedby \\frac{total\\ votes}{total\\ votes + l} \\newline \\text{Total accumulated evidence}\u0026amp; e_{accum} \\impliedby \\frac{total\\ positive\\ votes}{total\\ votes} \\newline \\text{Evidence per round} \u0026amp; e_{round} \\impliedby \\frac{round\\ positive\\ votes}{round\\ votes} \\newline \\end{array} $$
The node runs the new_votes and positive_votes parameters received in the query round through the following algorithm:
total_votes +== new_votes total_positive +== positive_votes confidence \u0026lt;-- total_votes / (total_votes + look_ahead) total_evidence \u0026lt;-- total_positive / total_votes new_evidence \u0026lt;-- positive_votes / new_votes evidence \u0026lt;-- new_evidence * ( 1 - confidence ) + total_evidence * confidence alpha \u0026lt;-- doubt * ( 1 - confidence ) + certainty * confidence Phase Three: Computation # In order to eliminate the need for a step function (a conditional in the code), we introduce a transition function from one regime to the other. Our interest in removing the step function is twofold:
Simplify the algorithm. With this change the number of branches is reduced, and everything is expressed as a set of equations.
The transition function makes the regime switch smooth, making it harder to potentially exploit the sudden regime change in some unforeseen manner. Such a swift change in operation mode could potentially result in a more complex behavior than initially understood, opening the door to elaborated attacks. The transition function proposed is linear with respect to the confidence.
Transition Function $$ \\begin{array}{cl} evidence \u0026amp; \\impliedby e_{round} (1 - c_{accum}) + e_{accum} c_{accum} \\newline \\alpha \u0026amp; \\impliedby \\alpha_1 (1 - c_{accum}) + \\alpha_2 c_{accum} \\newline \\end{array} $$
Since the confidence is modeled as a ratio that depends on the constant l, we can visualize the transition function at different values of l. Recall that this constant encapsulates the idea of “near future” in the frequentist certainty model: the higher it is, the more distant in time we consider the next valuable input of evidence to happen.
We have observed via experiment that for a transition function to be useful, we need establish two requirements:
The change has to be balanced and smooth, giving an opportunity to the first regime to operate and not jump directly to the second regime.
The convergence to 1.0 (fully operating in the second regime) should happen within a reasonable time-frame. We’ve set this time-frame experimentally at 1000 votes, which is in the order of ~100 queries given a k of 9.
[[ Note: Avalanche uses k = 20, as an experimental result from their deployment. Due to the fundamental similarities between the algorithms, it’s a good start for us. ]]
The node updates its local opinion on the consensus proposal by examining the relationship between the evidence accumulated for a proposal with the confidence encoded in the alpha parameter:
IF evidence \u0026gt; alpha THEN opinion \u0026lt;-- YES ELSE IF evidence \u0026lt; 1 - alpha THEN opinion \u0026lt;-- NO If the opinion of the node is NONE after evaluating the relation between evidence and alpha, adjust the number of uniform randomly queried nodes by multiplying the neighbors k by the k_multiplier up to the limit of k_max_multiplier_power query size increases.
;; possibly increase number nodes to uniformly randomly query in next round WHEN opinion is NONE AND k \u0026lt; k_original * k_multiplier ^ max_k_multiplier_power THEN k \u0026lt;-- k * k_multiplier Decision # The next step is a simple one: change our opinion if the threshold alpha is reached. This needs to be done separately for the YES/NO decision, checking both boundaries. The last step is then to decide on the current opinion. For that, a confidence threshold is employed. This threshold is derived from the network size, and is directly related to the number of total votes received.
Decision $$ \\begin{array}{cl} evidence \u0026gt; \\alpha \u0026amp; \\implies \\text{opinion YES} \\newline evidence \u0026lt; 1 - \\alpha \u0026amp; \\implies \\text{opinion NO} \\newline if\\ \\text{confidence} \u0026gt; c_{target} \u0026amp; THEN \\ \\text{finalize decision} \\newline \\end{array} $$
After the OPINION phase is executed, the current value of confidence is considered: if confidence exceeds a threshold derived from the network size and directly related to the total votes received, an honest node marks the decision as final, and always returns this opinion is response to further queries from other nodes on the network.
IF confidence \u0026gt; confidence_threshold OR round \u0026gt; max_rounds THEN finalized \u0026lt;-- T QUERY LOOP TERMINATES ELSE round +== 1 QUERY LOOP CONTINUES Thus, after the decision phase, either a decision has been finalized and the local node becomes quiescent never initiating a new query, or it initiates a new query.
Termination # A local round of Claro terminates in one of the following execution model considerations:
No queries are received for any newly initiated round for temporal periods observed via a locally computed passage of time. See the following point on local time.
The confidence on the proposal exceeds our threshold for finalization.
The number of rounds executed would be greater than max_rounds.
Quiescence # After a local node has finalized an opinion into a decision, it enters a quiescent state whereby it never solicits new votes on the proposal. The local node MUST reply with the currently finalized decision.
Clock # The algorithm only requires that nodes have computed the drift of observation of the passage of local time, not that that they have coordinated an absolute time with their peers. For an implementation of a phase locked-loop feedback to measure local clock drift see NTP.
Further points # Node receives information during round # In the query step, the node is envisioned as packing information into the query to cut down on the communication overhead a query to each of this k nodes containing the node\u0026rsquo;s own current opinion on the proposal (YES, NO, or NONE). The algorithm does not currently specify how a given node utilizes this incoming information. A possible use may be to count unsolicited votes towards a currently active round, and discard the information if the node is in a quiescent state.
Problems with Weighting Node Value of Opinions # If the view of other nodes is incomplete, then the sum of the optional weighting will not be a probability distribution normalized to 1.
The current algorithm doesn\u0026rsquo;t describe how the initial opinions are formed.
Implementation status # The following implementations have been created for various testing and simulation purposes:
Rust Python - FILL THIS IN WITH NEWLY CREATED REPO Common Lisp - FILL THIS IN WITH NEWLY CREATED REPO Wire Protocol # For interoperability we present a wire protocol semantics by requiring the validity of the following statements expressed in Notation3 (aka n3) about any query performed by a query node:
@prefix rdf: \u0026lt;http://www.w3.org/1999/02/22-rdf-syntax-ns#\u0026gt; . @prefix rdfs: \u0026lt;http://www.w3.org/2000/01/rdf-schema#\u0026gt; . @prefix xsd: \u0026lt;http://www.w3.org/2001/XMLSchema#\u0026gt; . @prefix Claro \u0026lt;https://rdf.logos.co/protocol/Claro#\u0026gt; . Claro:query :holds ( :_0 [ rdfs:label \u0026#34;round\u0026#34;; a xsd:postitiveInteger; ], rdfs:comment \u0026#34;\u0026#34;\u0026#34; The current round of this query A value of zero corresponds to the initial round. \u0026#34;\u0026#34;\u0026#34; ; :_1 [ rdfs:label \u0026#34;uri\u0026#34;; rdfs:comment \u0026#34;\u0026#34;\u0026#34; A unique URI for the proposal. It MAY be possible to examine the proposal by resolving this resource, and its associated URIs. \u0026#34;\u0026#34;\u0026#34; ; a xsd:anyURI ], :_2 [ rdfs:label \u0026#34;opinion\u0026#34;; rdfs:comment \u0026#34;\u0026#34;\u0026#34; The opinion on the proposal One of the strings \u0026#34;YES\u0026#34; \u0026#34;NO\u0026#34; or \u0026#34;NONE\u0026#34;. \u0026#34;\u0026#34;\u0026#34; ; # TODO constrain as an enumeration on three values efficiently a xsd:string ] ) . Nodes are advised to use Waku messages to include their own metadata in serializations as needed.
Syntax # The semantic description presented above can be reliably round-tripped through a suitable serialization mechanism. JSON-LD provides a canonical mapping to UTF-8 JSON.
At their core, the query messages are a simple enumeration of the three possible values of the opinion:
{ NO, NONE, YES } When represented via integers, such as choosing
{ -1, 0, +1 } the parity summations across network invariants often become easier to manipulate.
Security Considerations # Privacy # In practice, each honest node gossips its current opinion which reduces the number of messages that need to be gossiped for a given proposal. The resulting impact on the privacy of the node\u0026rsquo;s opinion is not currently analyzed.
Security with respect to various Adversarial Models # Adversarial models have been tested for which the values for current parameters of Claro have been tuned. Exposition of the justification of this tuning need to be completed.
Local Strategies # Random Adversaries # A random adversary optionally chooses to respond to all queries with a random decision. Note that this adversary may be in some sense Byzantine but not malicious. The random adversary also models some software defects involved in not \u0026ldquo;understanding\u0026rdquo; how to derive a truth value for a given proposition.
Infantile Adversary # Like a petulant child, an infantile adversary responds with the opposite vote of the honest majority on an opinion.
Omniscient Adversaries # Omniscient adversaries have somehow gained an \u0026ldquo;unfair\u0026rdquo; participation in consensus by being able to control f of N nodes with a out-of-band \u0026ldquo;supra-liminal\u0026rdquo; coordination mechanism. Such adversaries use this coordinated behavior to delay or sway honest majority consensus.
Passive Gossip Adversary # The passive network omniscient adversary is fully aware at all times of the network state. Such an adversary can always chose to vote in the most efficient way to block the distributed consensus from finalizing.
Active Gossip Adversary # An omniscient gossip adversary somehow not only controls f of N nodes, but has also has corrupted communications between nodes such that she may inspect, delay, and drop arbitrary messages. Such an adversary uses capability to corrupt consensus away from honest decisions to ones favorable to itself. This adversary will, of course, choose to participate in an honest manner until defecting is most advantageous.
Future Directions # Although we have proposed a normative description of the implementation of the underlying binary consensus algorithm (Claro), we believe we have prepared for analysis its adversarial performance in a manner that is amenable to replacement by another member of the snow* family.
We have presumed the existence of a general family of algorithms that can be counted on to vote on nodes in the DAG in a fair manner. Avalanche provides an example of the construction of votes on UTXO transactions. One can express all state machine, i.e. account-based models as checkpoints anchored in UTXO trust, so we believe that this presupposition has some justification. We can envision a need for tooling abstraction that allow one to just program the DAG itself, as they should be of stable interest no matter if Claro isn\u0026rsquo;t.
Informative References # Logos
On BFT Consensus Evolution: From Monolithic to DAG
snow-ipfs
snow* The Snow family of algorithms
Move Move: a Language for Writing DAG Abstractions
rdf
rdfs
xsd
n3-w3c-notes
ntp
Normative References # Claro
n3
json-ld
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:33,href:"/spec/4/",title:"4/MVDS-META",section:"Docs",content:`In this specification, we describe a method to construct message history that will aid the consistency guarantees of 2/MVDS. Additionally, we explain how data sync can be used for more lightweight messages that do not require full synchronization.
Motivation # In order for more efficient synchronization of conversational messages, information should be provided allowing a node to more effectively synchronize the dependencies for any given message.
Format # We introduce the metadata message which is used to convey information about a message and how it SHOULD be handled.
package vac.mvds; message Metadata { repeated bytes parents = 1; bool ephemeral = 2; } Nodes MAY transmit a Metadata message by extending the MVDS message with a metadata field.
message Message { bytes group_id = 6001; int64 timestamp = 6002; bytes body = 6003; + Metadata metadata = 6004; } Fields # Name Description parents list of parent message identifiers for the specific message. ephemeral indicates whether a message is ephemeral or not. Usage # parents # This field contains a list of parent message identifiers for the specific message. It MUST NOT contain any messages as parent whose ack flag was set to false. This establishes a directed acyclic graph (DAG)1 of persistent messages.
Nodes MAY buffer messages until dependencies are satisfied for causal consistency2, they MAY also pass the messages straight away for eventual consistency3.
A parent is any message before a new message that a node is aware of that has no children.
The number of parents for a given message is bound by [0, N], where N is the number of nodes participating in the conversation, therefore the space requirements for the parents field is O(N).
If a message has no parents it is considered a root. There can be multiple roots, which might be disconnected, giving rise to multiple DAGs.
ephemeral # When the ephemeral flag is set to false, a node MUST send an acknowledgment when they have received and processed a message. If it is set to true, it SHOULD NOT send any acknowledgment. The flag is false by default.
Nodes MAY decide to not persist ephemeral messages, however they MUST NOT be shared as part of the message history.
Nodes SHOULD send ephemeral messages in batch mode. As their delivery is not needed to be guaranteed.
Copyright # Copyright and related rights waived via CC0.
Footnotes # https://en.wikipedia.org/wiki/Directed_acyclic_graph\u0026#160;\u0026#x21a9;\u0026#xfe0e;
Jepsen. Causal Consistency. Jepsen, LLC.\u0026#160;\u0026#x21a9;\u0026#xfe0e;
https://en.wikipedia.org/wiki/Eventual_consistency\u0026#160;\u0026#x21a9;\u0026#xfe0e;
`}),e.add({id:34,href:"/spec/43/",title:"43/WAKU2-DEVICE-PAIRING",section:"Docs",content:` Abstract # In this document we describe a compound protocol for enabling two devices to mutually authenticate and securely exchange (arbitrary) information over the Waku network.
Background / Rationale / Motivation # In order to implement multi-device communications using one of the Noise session management mechanisms proposed in 37/WAKU2-NOISE-SESSIONS, we require a protocol to securely exchange (cryptographic) information between 2 or more devices possessed by a user.
Since, in this scenario, the devices would be close to each other, authentication can be initialized by exchanging a QR code out-of-band and then securely completed over the Waku network.
The protocol we propose consists of two main subprotocols or phases:
Device Pairing: two phisically close devices initialize the pairing by exchanging a QR code out-of-band. The devices then exchange and authenticate their respective long-term device ID static key by exchanging handshake messages over the Waku network; Secure Transfer: the devices securely exchange information in encrypted form using key material obtained during a successful pairing phase. The communication will happen over the Waku network, hence the devices do not need to be phisically close in this phase. Theory / Semantics # Device Pairing # In the pairing phase, device B requests to be paired to a device A. Once the two devices are paired, the devices will be mutually authenticated and will share a Noise session within which they can securely exchange information.
The request is made by exposing a QR code that, by default, has to be scanned by device A. If device A doesn\u0026rsquo;t have a camera while device B does, it is possible to execute a slightly different pairing (with same security guarantees), where A is exposing a QR code instead.
This protocol is designed in order to achieve two main security objectives:
resistance to Man-in-the-Middle attacks; provide network anonymity on devices\u0026rsquo; static keys, i.e. only paired devices will learn each other static key. Employed Cryptographic Primitives # H: the underlying cryptographically-secure hash function, e.g. SHA-256; HKDF: the key derivation function (based on H); Curve25519: the underlying elliptic curve for Diffie-Hellman (DH) operations. The WakuPairing Noise Handshake # The devices execute a custom handshake derived from XX, where they mutually exchange and authenticate their respective device static key by exchanging messages over the content topic with the following format
contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto The handshake, detailed in next section, can be summarized as:
WakuPairing: a. \u0026lt;- eB {H(sB||r), contentTopicParams, messageNametag} ... b. -\u0026gt; eA, eAeB {H(sA||s)} [authcode] c. \u0026lt;- sB, eAsB {r} d. -\u0026gt; sA, sAeB, sAsB {s} {}: payload, []: user interaction Protocol Flow # The device B exposes through a QR code a base64 (url safe) serialization of:
An ephemeral public key eB; The content topic parameters contentTopicParams = {application-name}, {application-version}, {shard-id}. A (randomly generated) 16-bytes long messageNametag. A commitment H(sB||r) for its static key sB where r is a random fixed-lenght value. The device A:
scans the QR code; obtains eB, contentTopicParams, messageNametag, Hash(sB||r); checks if {application-name} and {application-version} from contentTopicParams match the local application name and version: if not, aborts the pairing. Sets contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto; initializes the Noise handshake by passing contentTopicParams, messageNametag and Hash(sB||r) to the handshake prologue; executes the pre-handshake message, i.e. processes the key eB; executes the first handshake message over contentTopic, i.e. processes and sends a Waku message containing an ephemeral key eA; performs DH(eA,eB) (which computes a symmetric encryption key); attaches as payload to the handshake message the (encrypted) commitment H(sA||s) for A\u0026rsquo;s static key sA, where s is a random fixed-length value; an 8-digits authorization code authcode obtained as HKDF(h) mod 10^8 is displayed on the device, where h is the handshake hash value obtained once the first handshake message is processed. The device B:
sets contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto; listens to messages sent to contentTopic and locally filters only those with Waku payload starting with messageNametag. If any, continues. initializes the Noise handshake by passing contentTopicParams, messageNametag and Hash(sB||r) to the handshake prologue; executes the pre-handshake message, i.e. processes its ephemeral key eB; executes the first handshake message, i.e. obtains from the received message a public key eA. If eA is not a valid public key, the protocol is aborted. performs DH(eA,eB) (which computes a symmetric encryption key); decrypts the commitment H(sA||s) for A\u0026rsquo;s static key sA. an 8 decimal digits authorization code authcode obtained as HKDF(h) mod 10^8 is displayed on the device, where his the handshake hash value obtained once the first handshake message is processed. Device A and B wait for the user to confirm with an interaction (button press) that the authorization code displayed on both devices are the same. If not, the protocol is aborted.
The device B:
executes the second handshake message, i.e. processes and sends his (encrypted) device static key sB over contentTopic; performs DH(eA,sB) (which updates the symmetric encryption key); attaches as payload the (encrypted) commitment randomness r used to compute H(sB||r). The device A:
listens to messages sent to contentTopic and locally filters only those with Waku payload starting with messageNametag. If any, continues. decrypts the received message and obtains the public key sB. If sB is not a valid public key, the protocol is aborted. performs DH(eA,sB) (which updates a symmetric encryption key); decrypts the payload to obtain the randomness r. computes H(sB||r) and checks if this value corresponds to the commitment obtained in step 2. If not, the protocol is aborted. executes the third handshake message, i.e. processes and sends his (encrypted) device static key sA over contentTopic; performs DH(sA,eB) (which updates the symmetric encryption key); performs DH(sA,sB) (which updates the symmetric encryption key); attaches as payload the (encrypted) commitment randomness s used to compute H(sA||s). calls Split() and obtains two cipher states to encrypt inbound and outbound messages. The device B:
listens to messages sent to contentTopic and locally filters only those with Waku payload starting with messageNametag. If any, continues. obtains from decrypting the received message a public key sA. If sA is not a valid public key, the protocol is aborted. performs DH(sA,eB) (which updates a symmetric encryption key); performs DH(sA,sB) (which updates a symmetric encryption key); decrypts the payload to obtain the randomness s. Computes H(sA||s) and checks if this value corresponds to the commitment obtained in step 3. If not, the protocol is aborted. Calls Split() and obtains two cipher states to encrypt inbound and outbound messages. The WakuPairing for Devices without a Camera # In the above pairing handshake, the QR is by default exposed by device B and not by A because in most use-cases we foresee, the secure transfer phase would consist in exchanging a single message (e.g., Noise sessions, cryptographic keys, signatures, etc.) from device A to B.
However, since the user(s) confirm(s) at the end of message b. that the authorization code is the same on both devices, the role of the handhsake initiator and responder can be safely swapped in message a. and b..
Indeed, if the pairing phase successfully completes on both devices, the authentication code, the committed static keys and the Noise processing rules will ensure that no Man-in-the-Middle attack took place and that messages can be securely exchanged bi-directionally in the transfer phase.
This allows pairing in case device A does not have a camera to scan a QR (e.g. a desktop client) while device B has.
The resulting handshake would then be:
WakuPairing2: a. -\u0026gt; eA {H(sB||r), contentTopicParams, messageNametag} ... b. \u0026lt;- eB, eAeB {H(sB||r)} [authcode] c. \u0026lt;- sB, eAsB {r} d. -\u0026gt; sA, sAeB, sAsB {s} {}: payload, []: user interaction Secure Transfer # The pairing phase is designed to be application-agnostic and should be flexible enough to mutually authenticate and allow exchange of cryptographic key material between two devices over a distributed network of Waku2 nodes.
Once the handshake is concluded, (privacy-sensitive) information can be exchanged using the encryption keys agreed upon the pairing phase. If stronger security guarantees are required, some additional tweaks are possible.
Implementation Suggestions # Timebox QR exposure # We suggest to timebox the exposure of each pairing QR code to few seconds, e.g. 30. After this time limit, a QR code containing a new ephemeral key, random static key commitment and message nametag (content topic parameters could remain the same) should replace the previously exposed QR, which can then be discarded.
The reason for such suggestion is due to the fact that if an attacker is able to compromise one of the ephemeral keys, he might successfully realize an undetected MitM attack up to the authcode confirmation (we note that compromising ephemeral keys is outside our and Noise security assumptions).
The attacker could indeed proceed as follows:
intercepts the QR; blocks/delays the delivery of the pairing message b.; compromises A or B ephemeral key; recovers the genuine authcode that would have been generated by A and B; generates ~10^8 random t values until the Noise processing of the message b'. -\u0026gt; eC, eCeB {H(sC||t)} , where eC and sC are the attacker ephemeral and static key, respectively, results in computing the same authcode as the one between A and B; delivers the message b'. -\u0026gt; eC, eCeB {H(sC||t)} to B (before A is able to deliver its message b.). At this point A and B will observe the same authcode (and would then confirm it), but B will process the attacker\u0026rsquo;s ephemeral key eC instead of eA.
However, the attacker would not be able to open to device A the static key commitment H(sB||s) sent by device B out-of-band, and the pairing will abort on A side before it reveals its static key. Device B, instead, will successfully complete the pairing with the attacker.
Hence, timeboxing the QR exposure, also in combination with increasing the number of decimal digits of the authcode, will strongly limit the probability that an attacker can successfully impersonate device A to B.
We stress once more, that such attack requires the compromise of an ephemeral key (outside our security model) and that device A will in any case detect a mismatch and abort the pairing, regardless of the fact that the QR timeboxing mitigation is implemented or not.
Randomized Rekey # The Noise Protocol framework supports Rekey() in order to update encryption keys \u0026ldquo;so that a compromise of cipherstate keys will not decrypt older [exchanged] messages\u0026rdquo;. However, if a certain cipherstate key is compromised, it will be possible for the attacker not only to decrypt messages encrypted under that key, but also all those messages encrypted under any successive new key obtained through a call to Rekey().
This could be mitigated by attaching an ephemeral key to messages sent after a Split() so that a new random symmetric key can be derived, in a similar fashion to Double-Ratchet.
This can be practically achieved by:
keeping the full Handhshake State even after the handshake is complete (by Noise specification a call to Split() should delete the Handshake State) continuing updating the Handshake State by processing every after-handshake exchanged message (i.e. the payload) according to the Noise processing rules (i.e. by calling EncryptAndHash(payload) and DecryptAndHash(payload)); adding to each (or every few) message exchanged in the transfer phase a random ephemeral key e and perform Diffie-Hellman operations with the other party\u0026rsquo;s ephemeral/static keys in order to update the underlying CipherState and recover new random inbound/outbound encryption keys by calling Split(). In short, the transfer phase would look like (but not necessarily the same as):
TransferPhase: -\u0026gt; eA, eAeB, eAsB {payload} \u0026lt;- eB, eAeB, sAeB {payload} ... {}: payload Messages Nametag Derivation # To reduce metadata leakages and increase devices\u0026rsquo;s anonymity over the p2p network, 35/WAKU2-NOISE suggests to use some common secrets mntsInbound, mntsOutbound (e.g. mntsInbound, mntsOutbound = HKDF(h) where h is the handshake hash value of the Handshake State at some point of the pairing phase) in order to frequently and deterministically change the messageNametag of messages exchanged during the pairing and transfer phase - ideally, at each message exchanged.
Given the proposed construction, the mntsInbound and mntsOutbound secrets can be used to iteratively generate the messageNametag field of Waku payloads for inbound and outbound messages, respectively.
The derivation of messageNametag should be deterministic only for communicating devices and independent from message content, otherwise lost messages will prevent computing the next message nametag. A possible approach consists in computing the n-th messageNametag as H( mntsInbound || n), where n is serialized as uint64.
In this way, sender\u0026rsquo;s and recipient\u0026rsquo;s devices can keep updated a buffer of messageNametag to sieve while listening to messages sent over /{application-name}/{application-version}/wakunoise/1/sessions-{shard-id}/ (i.e., the next 50 not yet seen). They will then be able to further identify if one or more messages were eventually lost or not-yet-delivered during the communication. This approach brings also the advantage that communicating devices can efficiently identify encrypted messages addressed to them.
We note that since the ChaChaPoly cipher used to encrypt messages supports additional data, an encrypted payload can be further authenticated by passing the messageNametag as additional data to the encryption/decryption routine. In this way, an attacker would be unable to craft an authenticated Waku message even in case the currently used symmetric encryption key is compromised, unless mntsInbound, mntsOutbound or the messageNametag buffer lists were compromised too.
Security/Privacy Considerations # Assumptions # The attacker is active, i.e. can interact with both devices A and B by sending messages over contentTopic.
The attacker has access to the QR code, that is knows the ephemeral key eB, the commitment H(sB||r) and the contentTopic exposed by the device B.
Devices A and B are considered trusted (otherwise the attacker will simply exfiltrate the relevant information from the attacked device).
As common for Noise, we assume that ephemeral keys cannot be compromised, while static keys might be later compromised. However, we enforce in the pairing phase extra security mechanisms (i.e. use of commitments for static keys) that will prevent some attacks possible when ephemeral keys are weak or get compromised.
Rationale # The device B exposes a commitment to its static key sB because:
it can commit to its static key before the authentication code is confirmed without revealing it. If the private key of eB is weak or gets compromised, an attacker can impersonate B by sending in message c. to device A his own static key and successfully complete the pairing phase. Note that being able to compromise eB is not contemplated by our security assumptions. B cannot adaptively choose a static key based on the state of the Noise handshake at the end of message b., i.e. after the authentication code is confirmed. Note that device B is trusted in our security assumptions. Confirming the authentication code after processing message b. will ensure that no Man-in-the-Middle (MitM) can later send a static key different than sB. The device A sends a commitment to its static key sA because:
it can commit to its static key before the authentication code is confirmed without revealing it. A cannot adaptively choose a static key based on the state of the Noise handshake at the end of message b., i.e. after the authentication code is confirmed. Note that device A is trusted in our security assumptions. Confirming the authentication code after processing message b. will ensure that no MitM can later send a static key different than sA. The authorization code is shown and has to be confirmed at the end of message b. because:
an attacker that frontruns device A by sending faster his own ephemeral key would be detected before he\u0026rsquo;s able to know device B static key sB; it ensures that no MitM attacks will happen during the whole pairing handshake, since commitments to the (later exchanged) device static keys will be implicitly acknowledged by the authorization code confirmation; it enables to safely swap the role of handshake initiator and responder (see above); Device B sends his static key first because:
by being the pairing requester, it cannot probe device A identity without revealing its own (static key) first. Note that device B static key and its commitment can be bound to other cryptographic material (e.g., seed phrase). Device B opens a commitment to its static key at message c. because:
if device A replies concluding the handshake according to the protocol, device B acknowledges that device A correctly received his static key sB, since r was encrypted under an encryption key derived from the static key sB and the genuine (due to the previous authcode verification) ephemeral keys eA and eB. Device A opens a commitment to its static key at message d. because:
if device B doesn\u0026rsquo;t abort the pairing, device A acknowledges that device B correctly received his static key sA, since s was encrypted under an encryption key derived from the static keys sA and sB and the genuine (due to the previous authcode verification) ephemeral keys eA and eB. Application to Noise Sessions # The N11M session management mechanism # In the N11M session management mechanism, one of Alice\u0026rsquo;s devices is already communicating with one of Bob\u0026rsquo;s devices within an active Noise session, e.g. after a successful execution of a Noise handshake.
Alice and Bob would then share some cryptographic key material, used to encrypt their communications. According to 37/WAKU2-NOISE-SESSIONS this information consists of:
A session-id (32 bytes) Two cipher state CSOutbound, CSInbound, where each of them contains: an encryption key k (2x32bytes) a nonce n (2x8bytes) (optionally) an internal state hash h (2x32bytes) for a total of 176 bytes of information.
In a N11M session mechanism scenario, all (synced) Alice\u0026rsquo;s devices that are communicating with Bob share the same Noise session cryptographic material. Hence, if Alice wishes to add a new device, she must securely transfer a copy of such data from one of her device A to a new device B in her possession.
In order to do so she can:
pair device A with B in order to have a Noise session between them; securely transfer within such session the 176 bytes serializing the active session with Bob; manually instantiate in B a Noise session with Bob from the received session serialization. Copyright # Copyright and related rights waived via CC0.
References # Normative # 35/WAKU2-NOISE 37/WAKU2-NOISE-SESSIONS Informative # 26/WAKU2-PAYLOAD The Double-Ratchet Algorithm The Noise Protocol Framework specifications IETF RFC 4648 - The Base16, Base32, and Base64 Data Encodings `}),e.add({id:35,href:"/spec/44/",title:"44/WAKU2-DANDELION",section:"Docs",content:` Abstract # This document specifies a deanonymization mitigation technique, based on Dandelion and Dandelion++, for Waku Relay. It mitigates mass deanonymization in the multi-node (botnet) attacker model, even when the number of malicious nodes is linear in the number of total nodes in the network.
Based on the insight that symmetric message propagation makes deanonymization easier, it introduces a probability for nodes to simply forward the message to one select relay node instead of disseminating messages as per usual relay operation.
Background and Motivation # Waku Relay, offers privacy, pseudonymity, and a first layer of anonymity protection by design. Being a modular protocol family Waku v2 offers features that inherently carry trade-offs as separate building blocks. Anonymity protection is such a feature. The Anonymity Trilemma states that an anonymous communication network can only have two out of low bandwidth consumption, low latency, and strong anonymity. Even when choosing low bandwidth and low latency, which is the trade-off that basic Waku Relay takes, better anonymity properties (even though not strong per definition) can be achieved by sacrificing some of the efficiency properties. 44/WAKU2-DANDELION specifies one such technique, and aims at gaining the best \u0026ldquo;bang for the buck\u0026rdquo; in terms of efficiency paid for anonymity gain. 44/WAKU2-DANDELION is based on Dandelion and Dandelion++.
Dandelion is a message spreading method, which, compared to other methods, increases the uncertainty of an attacker when trying to link messages to senders. Libp2p gossipsub aims at spanning a d-regular graph topology, with d=6 as the default value. Messages are forwarded within this (expected) symmetric topology, which reduces uncertainty when trying to link messages to senders. Dandelion breaks this symmetry by subdividing message spreading into a \u0026ldquo;stem\u0026rdquo; and a \u0026ldquo;fluff\u0026rdquo; phase.
In the \u0026ldquo;stem\u0026rdquo; phase, the message is sent to a single select relay node. With a certain probability, this message is relayed further on the \u0026ldquo;stem\u0026rdquo;, or enters the fluff phase. On the stem, messages are relayed to single peers, respectively, while in fluff phase, messages are spread as per usual relay operation (optionally augmented by random delays to further reduce symmetry). The graph spanned by stem connections is referred to as the anonymity graph.
Note: This is an early raw version of the specification. It does not strictly follow the formally evaluated Dandelion++ paper, as we want to experiment with relaxing (and strengthening) certain model properties. In specific, we aim at a version that has tighter latency bounds. Further research suggests that Dandelion++\u0026rsquo;s design choices are not optimal, which further assures that tweaking design choices makes sense. We will refine design decisions in future versions of this specification.
Further information on Waku anonymity may be found in our Waku Privacy and Anonymity Analysis.
Theory and Functioning # 44/WAKU2-DANDELION can be seen as an anonymity enhancing add-on to Waku Relay message dissemination, which is based on libp2p gossipsub. 44/WAKU2-DANDELION subdivides message dissemination into a \u0026ldquo;stem\u0026rdquo; and a \u0026ldquo;fluff\u0026rdquo; phase. This specification is mainly concerned with specifying the stem phase. The fluff phase corresponds to Waku Relay, with optional fluff phase augmentations such as random delays. Adding random delay in the fluff phase further reduces symmetry in dissemination patterns and introduces more uncertainty for the attacker. Specifying fluff phase augmentations is out of scope for this document.
Note: We plan to add a separate specification for fluff phase augmentations. We envision stem and fluff phase as abstract concepts. The Dandelion stem and fluff phases instantiate these concepts. Future stem specifications might comprise: none (standard relay), Dandelion stem, Tor, and mix-net. As for future fluff specifications: none (standard relay), diffusion (random delays), and mix-net.
Messages relayed by nodes supporting 44/WAKU2-DANDELION are either in stem phase or in fluff phase. We refer to the former as a stem message and to the latter as a fluff message. A message starts in stem phase, and at some point, transitions to fluff phase. Nodes, on the other hand, are in stem state or fluff state. Nodes in stem state relay stem messages to a single relay node, randomly selected per epoch for each incoming stem connection. Nodes in fluff state transition stem messages into fluff phase and relay them accordingly. Fluff messages are always disseminated via Waku Relay, by both nodes in stem state and nodes in fluff state.
Messages originated in the node (i.e. messages coming from the application layer of our node), are always sent as stem messages.
The stem phase can be seen as a different protocol, and messages are introduced into Waku Relay, and by extension gossipsub, once they arrive at a node in fluff state for the first time. 44/WAKU2-DANDELION uses 19/WAKU2-LIGHTPUSH as the protocol for relaying stem messages.
There are no negative effects on gossipsub peer scoring, because Dandelion nodes in stem state still normally relay Waku Relay (gossipsub) messages.
Specification # Nodes $v$ supporting 44/WAKU2-DANDELION MUST either be in stem state or in fluff state. This does not include relaying messages originated in $v$, for which $v$ SHOULD always be in stem state.
Choosing the State # On startup and when a new epoch starts, node $v$ randomly selects a number $r$ between 0 and 1. If $r \u0026lt; q$, for $q = 0.2$, the node enters fluff state, otherwise, it enters stem state.
New epochs start when unixtime (in seconds) $\\equiv 0 \\mod 600$, corresponding to 10 minute epochs.
Stem State # On entering stem state, nodes supporting 44/WAKU2-DANDELION MUST randomly select two nodes for each pubsub topic from the respective gossipsub mesh node set. These nodes are referred to as stem relays. Stem relays MUST support 19/WAKU2-LIGHTPUSH. If a chosen peer does not support 19/WAKU2-LIGHTPUSH, the node SHOULD switch to fluff state. (We may update this strategy in future versions of this document.)
Further, the node establishes a map that maps each incoming stem connection to one of its stem relays chosen at random (but fixed per epoch). Incoming stem connections are identified by the Peer IDs of peers the node receives 19/WAKU2-LIGHTPUSH messages from. Incoming 19/WAKU2-LIGHTPUSH connections from peers that do not support 44/WAKU2-DANDELION are identified and mapped in the same way. This makes the protocol simpler, increases the anonymity set, and offers Dandelion anonymity properties to such peers, too.
The node itself is mapped in the same way, so that all messages originated by the node are relayed via a per-epoch-fixed Dandelion relay, too.
While in stem state, nodes MUST relay stem messages to the respective stem relay. Received fluff messages MUST be relayed as specified in the fluff state section.
The stem protocol (19/WAKU2-LIGHTPUSH) is independent of the fluff protocol (Waku Relay). While in stem state, nodes MUST NOT gossip about stem messages, and MUST NOT send control messages related to stem messages. (An existing gossipsub implementation does not have to be adjusted to not send gossip about stem messages, because these messages are only handed to gossipsub once they enter fluff phase.)
Fail Safe # Nodes $v$ in stem state SHOULD store messages attached with a random timer between $t_1 = 5 * 100ms$ and $t_2 = 2 * t_1$. This time interval is chosen because
we assume $100,ms$ as an average per hop delay, and using $q=0.2$ will lead to an expected number of 5 stem hops per message. If $v$ does not receive a given message via Waku Relay (fluff) before the respective timer runs out, $v$ will disseminate the message via Waku Relay.
Fluff State # In fluff state, nodes operate as usual Waku Relay nodes. The Waku Relay functionality might be augmented by a future specification, e.g. adding random delays.
Note: The Dandelion paper describes the fluff phase as regular forwarding. Since Dandelion is designed as an update to the Bitcoin network using diffusion spreading, this regular forwarding already comprises random delays.
Implementation Notes # Handling of the 44/WAKU2-DANDELION stem phase can be implemented as an extension to an existing 19/WAKU2-LIGHTPUSH implementation.
Fluff phase augmentations might alter gossipsub message dissemination (e.g. adding random delays). If this is the case, they have to be implemented on the libp2p gossipsub layer.
Security/Privacy Considerations # Denial of Service: Black Hole Attack # In a black hole attack, malicious nodes prevent messages from being spread, metaphorically not allowing messages to leave once they entered. This requires the attacker to control nodes on all dissemination paths. Since the number of dissemination paths is significantly reduced in the stem phase, Dandelion spreading reduces the requirements for a black hole attack.
The fail-safe mechanism specified in this document (proposed in the Dandelion paper), mitigates this.
Anonymity Considerations # Attacker Model and Anonymity Goals # 44/WAKU2-DANDELION provides significant mitigation against mass deanonymization in the passive scaling multi node model. in which the attacker controls a certain percentage of nodes in the network. 44/WAKU2-DANDELION provides significant mitigation against mass deanonymization even if the attacker knows the network topology, i.e. the anonymity graph and the relay mesh graph.
Mitigation in stronger models, including the active scaling multi-node model, is weak. We will elaborate on this in future versions of this document.
44/WAKU2-DANDELION does not protect against targeted deanonymization attacks.
Non-Dandelion Peers # Stem relays receiving messages can either be in stem state or in fluff state themselves. They might also not support 44/WAKU2-DANDELION, and interpret the message as classical 19/WAKU2-LIGHTPUSH, which effectively makes them act as fluff state relays. While such peers lower the overall anonymity properties, the Dandelion++ paper showed that including those peers yields more anonymity compared to excluding these peers.
Future Analysis # The following discusses potential relaxations in favour of reduced latency, as well as their impact on anonymity. This is still work in progress and will be elaborated on in future versions of this document.
Generally, there are several design choices to be made for the stem phase of a Dandelion-based specification:
the probability of continuing the stem phase, which determines the expected stem lengh, the out degree in the stem phase, which set to 1 in this document (also in the Dandelion papers), the rate of re-selecting stem relays among all gossipsub mesh peers (for a given pubsub topic), and the mapping of incoming connections to outgoing connections. Bound Stem Length # Choosing $q = 0.2$, 44/WAKU2-DANDELION has an expected stem length of 5 hops, Assuming $100ms$ added delay per hop, the stem phase adds around 500ms delay on average.
There is a possibility for the stem to grow longer, but some applications need tighter bounds on latency.
While fixing the stem length would yield tighter latency bounds, it also reduces anonymity properties. A fixed stem length requires the message to carry information about the remaining stem length. This information reduces the uncertainty of attackers when calculating the probability distribution assigning each node a probability for having sent a specific message. We will quantify the resulting loss of anonymity in future versions of this document.
Stem Relay Selection # In its current version, 44/WAKU2-DANDELION nodes default to fluff state if the random stem relay selection yields at least one peer that does not support 19/WAKU2-LIGHTPUSH (which is the stem protocol used in 44/WAKU2-DANDELION). If nodes would reselect peers until they find peers supporting 19/WAKU2-LIGHTPUSH, malicious nodes would get an advantage if a significant number of honest nodes would not support 19/WAKU2-LIGHTPUSH. Even though this causes messages to enter fluff phase earlier, we choose the trade-off in favour of protocol stability and sacrifice a bit of anonymity. (We will look into improving this in future versions of this document.)
Random Delay in Fluff Phase # Dandelion and Dandelion++ assume adding random delays in the fluff phase as they build on Bitcoin diffusion. 44/WAKU2-DANDELION (in its current state) allows for zero delay in the fluff phase and outsources fluff augmentations to dedicated specifications. While this lowers anonymity properties, it allows making Dandelion an opt-in solution in a given network. Nodes that do not want to use Dandelion do not experience any latency increase. We will quantify and analyse this in future versions of this specification.
We plan to add a separate fluff augmentation specification that will introduce random delays. Optimal delay times depend on the message frequency and patterns. This delay fluff augmentation specification will be oblivious to the actual message content, because Waku Dandelion specifications add anonymity on the routing layer. Still, it is important to note that Waku2 messages (in their current version) carry an originator timestamp, which works against fluff phase random delays. An analysis of the benefits of this timestamp versus anonymity risks is on our roadmap.
By adding a delay, the fluff phase modifies the behaviour of libp2p gossipsub, which Waku Relay builds upon.
Note: Introducing random delays can have a negative effect on peer scoring.
Stem Flag # While 44/WAKU2-DANDELION without fluff augmentation does not effect Waku Relay nodes, messages sent by nodes that only support 19/WAKU2-LIGHTPUSH might be routed through a Dandelion stem without them knowing. While this improves anonymity, as discussed above, it also introduces additional latency and lightpush nodes cannot opt out of this.
In future versions of this specification we might
add a flag to 14/WAKU2-MESSAGE indicating a message should be routed over a Dandelion stem (opt-in), or add a flag to 14/WAKU2-MESSAGE indicating a message should not be routed over a Dandelion stem (opt-out), or introducing a fork of 19/WAKU2-LIGHTPUSH exclusively used for Dandelion stem. In the current version, we decided against these options in favour of a simpler protocol and an increased anonymity set.
Copyright # Copyright and related rights waived via CC0.
References # Dandelion Dandelion++ Waku Relay Waku v2 d-regular graph Anonymity Trilemma Waku Privacy and Anonymity Analysis. On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies Adversarial Models 14/WAKU2-MESSAGE `}),e.add({id:36,href:"/spec/45/",title:"45/WAKU2-ADVERSARIAL-MODELS",section:"Docs",content:" Abstract # This document lists adversarial models and attack-based threats relevant in the context of Waku v2.\nMotivation and Background # Future versions of this document will serve as a comprehensive list of adversarial models and attack based threats relevant for Waku v2. The main purpose of this document is being a linkable resource for specifications that address protection as well as mitigation mechanisms within the listed models.\nDiscussing and introducing countermeasures to specific attacks in specific models is out of scope for this document. Analyses and further information about Waku\u0026rsquo;s properties within these models may be found in our Waku v2 Anonymity Analysis series of research log posts:\nPart I: Definitions and Waku Relay Note: This document adds to the adversarial models and threat list discussed in our research log post. It does not cover analysis of Waku, as the research log post does. Future versions of this document will extend the adversarial models and threat list.\nInformal Definitions: Security, Privacy, and Anonymity # The concepts of security, privacy, and anonymity are linked and have quite a bit of overlap.\nSecurity # Of the three, Security has the clearest agreed upon definition, at least regarding its key concepts: confidentiality, integrity, and availability.\nconfidentiality: data is not disclosed to unauthorized entities. integrity: data is not modified by unauthorized entities. availability: data is available, i.e. accessible by authorized entities. While these are the key concepts, the definition of information security has been extended over time including further concepts, e.g. authentication and non-repudiation.\nPrivacy # Privacy allows users to choose which data and information\nthey want to share and with whom they want to share it. This includes data and information that is associated with and/or generated by users. Protected data also comprises metadata that might be generated without users being aware of it. This means, no further information about the sender or the message is leaked. Metadata that is protected as part of the privacy-preserving property does not cover protecting the identities of sender and receiver. Identities are protected by the anonymity property.\nOften privacy is realized by the confidentiality property of security. This neither makes privacy and security the same, nor the one a sub category of the other. While security is abstract itself (its properties can be realized in various ways), privacy lives on a more abstract level using security properties. Privacy typically does not use integrity and availability. An adversary who has no access to the private data, because the message has been encrypted, could still alter the message.\nAnonymity # Privacy and anonymity are closely linked. Both the identity of a user and data that allows inferring a user\u0026rsquo;s identity should be part of the privacy policy. For the purpose of analysis, we want to have a clearer separation between these concepts.\nWe define anonymity as unlinkablity of users\u0026rsquo; identities and their shared data and/or actions.\nWe subdivide anonymity into receiver anonymity and sender anonymity.\nReceiver Anonymity # We define receiver anonymity as unlinkability of users\u0026rsquo; identities and the data they receive and/or related actions. Because each Waku message is associated with a content topic, and each receiver is interested in messages with specific content topics, receiver anonymity in the context of Waku corresponds to subscriber-topic unlinkability. An example for the \u0026ldquo;action\u0026rdquo; part of our receiver anonymity definition is subscribing to a specific topic.\nSender Anonymity # We define sender anonymity as unlinkability of users\u0026rsquo; identities and the data they send and/or related actions. Because the data in the context of Waku is Waku messages, sender anonymity corresponds to sender-message unlinkability.\nAnonymity Trilemma # The Anonymity trilemma states that only two out of strong anonymity, low bandwidth, and low latency can be guaranteed in the global attacker model. Waku\u0026rsquo;s goal, being a modular set of protocols, is to offer any combination of two out of these three properties, as well as blends.\nA fourth factor that influences the anonymity trilemma is frequency and patterns of messages. The more messages there are, and the more randomly distributed they are, the better the anonymity protection offered by a given anonymous communication protocol. So, incentivising users to use the protocol, for instance by lowering entry barriers, helps protecting the anonymity of all users. The frequency/patterns factor is also related to k-anonymity.\nCensorship Resistance # Another security related property that Waku aims to offer is censorship resistance. Censorship resistance guarantees that users can participate even if an attacker tries to deny them access. So, censorship resistance ties into the availability aspect of security. In the context of Waku that means users should be able to send messages as well as receive all messages they are interested in, even if an attacker tries to prevent them from disseminating messages or tries to deny them access to messages.\nAn example for a censorship resistance technique is Tor\u0026rsquo;s Pluggable Transports.\nAdversarial Models # The following lists various attacker types with varying degrees of power. The more power an attacker has, the more difficult it is to gain the respective attacker position.\nEach attacker type comes in a passive and an active variant. While a passive attacker can stay hidden and is not suspicious, the respective active attacker has more (or at least the same) deanonymization power.\nWe also distinguish between internal and external attackers. Since in permissionless protocols it is easy to obtain an internal position, in practice attackers are expected to mount combined attacks that leverage both internal and external attacks.\nInternal # In the passive variant, an internal attacker behaves like an honest node towards peers. The passive internal attacker has the same access rights as any honest node. In the active variant, an internal attacker can additionally drop, inject, and alter messages. With respect to Waku relay, for example, an internal attacker participates in the same pubsub topic as its victims, and can read messages related to that topic.\nSingle Node # This attacker controls a single node.\nMulti Node # This attacker controls a fixed number of nodes (not scaling with the total number of nodes in the network). The multi node position can be achieved by setting up multiple nodes. Botnets might be leveraged to increase the number of available hosts. Multi node attackers could use Sybil attacks to increase the number of controlled nodes. A countermeasure is for nodes to only accept libp2p gossipsub graft requests from peers with different IP addresses, or even different subnets.\nNodes controlled by the attacker can efficiently communicate out-of-band to coordinate.\nScaling Multi Node # This attacker controls a number of nodes that scales linearly with the number of nodes in the network. The attacker controls $p%$ of all nodes in the network.\nNodes controlled by the attacker can efficiently communicate out-of-band to coordinate.\nExternal # An external attacker can only see encrypted traffic. Waku protocols are protected by a secure channel set up with Noise.\nLocal # A local attacker has access to communication links in a local network segment. This could be a rogue access point (with routing capability).\nAS # An AS attacker controls a single AS (autonomous system). A passive AS attacker can listen to traffic on arbitrary links within the AS. An active AS attacker can drop, delay, inject, and alter traffic on arbitrary links within the AS.\nIn practice, a malicious ISP would be considered as an AS attacker. A malicious ISP could also easily setup a set of nodes at specific points in the network, gaining internal attack power similar to a strong multi node or even scaling multi node attacker.\nGlobal (On-Net) # A global (on-net) attacker has complete overview over the whole network. A passive global attacker can listen to traffic on all links, while the active global attacker basically carries the traffic: it can freely drop, delay, inject, and alter traffic at all positions in the network. This basically corresponds to the Dolev-Yao model.\nAn entity with this power would, in practice, also have the power of the internal scaling multi node attacker.\nAttack-based Threats # The following lists various attacks against Waku v2 protocols. If not specifically mentioned, the attacks refer to Waku relay and the underlying libp2p GossipSub. We also list the weakest attacker model in which the attack can be successfully performed against.\nAn attack is considered more powerful if it can be successfully performed in a weaker attacker model.\nNote: This list is work in progress. We will either expand this list adding more attacks in future versions of this document, or remove it and point to the \u0026ldquo;Security Considerations\u0026rdquo; sections of respective RFCs.\nPrerequisite: Get a Specific Position in the Network # Some attacks require the attacker node(s) to be in a specific position in the network. In most cases, this corresponds to trying to get into the mesh peer list for the desired pubsub topic of the victim node.\nIn libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a graft message for the desired topic to the victim node. If the victim node still has open slots, the attacker gets the desired position. This only requires the attacker to know the gossipsub multiaddress of the victim node.\nA scaling multi node attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes.\nSender Deanonymization # This section lists attacks that aim at deanonymizing a message sender.\nWe assume that protocol messages are transmitted within a secure channel set up using the Noise Protocol Framework. For Waku Relay this means we only consider messages with version field 2, which indicates that the payload has to be encoded using 35/WAKU2-NOISE.\nNote: The currently listed attacks are against libp2p in general. The data field of Waku v2 relay must be a Waku v2 message. The attacks listed in the following do not leverage that fact.\nReplay Attack # In a replay attack, the attacker replays a valid message it received.\nWaku relay is inherently safe against replay attack, because GossipSub nodes, and by extension Waku relay nodes, feature a seen cache, and only relay messages they have not seen before.\nFurther, replay attacks will be punished by RLN Relay.\nObserving Messages # If Waku relay was not protected with Noise, the AS attacker could simply check for messages leaving $v$ which have not been relayed to $v$. These are the messages sent by $v$. Waku relay protects against this attack by employing secure channels setup using Noise.\nNeighbourhood Surveillance # This attack can be performed by a single node attacker that is connected to all peers of the victim node $v$ with respect to a specific topic mesh. The attacker also has to be connected to $v$. In this position, the attacker will receive messages $m_v$ sent by $v$ both on the direct path from $v$, and on indirect paths relayed by peers of $v$. It will also receive messages $m_x$ that are not sent by $v$. These messages $m_x$ are relayed by both $v$ and the peers of $v$. Messages that are received (significantly) faster from $v$ than from any other of $v$\u0026rsquo;s peers are very likely messages that $v$ sent, because for these messages the attacker is one hop closer to the source.\nThe attacker can (periodically) measure latency between itself and $v$, and between itself and the peers of $v$ to get more accurate estimates for the expected timings. An AS attacker (and if the topology allows, even a local attacker) could also learn the latency between $v$ and its well-behaving peers. An active AS attacker could also increase the latency between $v$ and its peers to make the timing differences more prominent. This, however, might lead to $v$ switching to other peers.\nThis attack cannot (reliably) distinguish messages $m_v$ sent by $v$ from messages $m_y$ relayed by peers of $v$ the attacker is not connected to. Still, there are hop-count variations that can be leveraged. Messages $m_v$ always have a hop-count of 1 on the path from $v$ to the attacker, while all other paths are longer. Messages $m_y$ might have the same hop-count on the path from $v$ as well as on other paths. Further techniques that are part of the mass deanonymization category, such as bayesian analysis, can be used here as well.\nControlled Neighbourhood # If a multi node attacker manages to control all peers of the victim node, it can trivially tell which messages originated from $v$.\nCorrelation # Monitoring all traffic (in an AS or globally), allows the attacker to identify traffic correlated with messages originating from $v$. This (alone) does not allow an external attacker to learn which message $v$ sent, but it allows identifying the respective traffic propagating through the network. The more traffic in the network, the lower the success rate of this attack.\nCombined with just a few nodes controlled by the attacker, the actual message associated with the correlated traffic can eventually be identified.\nMass Deanonymization # While attacks in the sender deanonymization category target a set of either specific or arbitrary users, attacks in the mass deanonymization category aim at deanonymizing (parts of) the whole network. Mass deanonymization attacks do not necessarily link messages to senders. They might only reduce the anonymity set in which senders hide, or infer information about the network topology.\nGraph Learning # Graph learning attacks are a prerequisite for some mass deanonymization attacks, in which the attacker learns the overlay network topology. Graph learning attacks require a scaling multinode attacker\nFor gossipsub this means an attacker learns the topic mesh for specific pubsub topics. Dandelion++ describes ways to perform this attack.\nBayesian Analysis # Bayesian analysis allows attackers to assign each node in the network a likelihood of having sent (originated) a specific message. Bayesian analysis for mass deanonymization is detailed in On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies. It requires a scaling node attacker as well as knowledge of the network topology, which can be learned via graph learning attacks.\nDenial of Service (DoS) # Flooding # In a flooding attack, attackers flood the network with bogus messages.\nWaku employs RLN Relay as the main countermeasure to flooding. SWAP also helps mitigating DoS attacks.\nBlack Hole (internal) # In a black hole attack, the attacker does not relay messages it is supposed to relay. Analogous to a black hole, attacker nodes do not allow messages to leave once they entered.\nWhile single node and smaller multi node attackers can have a negative effect on availability, the impact is not significant. A scaling multi node attacker, however, can significantly disrupt the network with such an attack.\nThe effects of this attack are especially severe in conjunction with deanonymization mitigation techniques that reduce the out-degree of the overlay, such as Waku Dandelion. (Waku Dandelion also discusses mitigation techniques compensating the amplified black hole potential.)\nTraffic Filtering (external) # A local attacker can filter and drop all Waku traffic within its controlled network segment. An AS attacker can filter and drop all Waku traffic within its authority, while a global attacker can censor the whole network. A countermeasure are censorship resistance techniques like Pluggable Transports.\nAn entity trying to censor Waku can employ both the black hole attack and traffic filtering; the former is internal while the latter is external.\nCopyright # Copyright and related rights waived via CC0.\nReferences # 10/WAKU2 11/WAKU2-RELAY libp2p GossipSub Security Authentication Anonymity Trilemma Waku v2 message Pluggable Transports Sybil attack Dolev-Yao model Noise Protocol Framework 35/WAKU2-NOISE 17/WAKU-RLN-RELAY 18/WAKU2-SWAP Dandelion++ On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies "}),e.add({id:37,href:"/spec/46/",title:"46/GOSSIPSUB-TOR-PUSH",section:"Docs",content:` Abstract # This document extends the libp2p gossipsub specification specifying gossipsub Tor Push, a gossipsub-internal way of pushing messages into a gossipsub network via Tor. Tor Push adds sender identity protection to gossipsub.
Protocol identifier: /meshsub/1.1.0
Note: Gossipsub Tor Push does not have a dedicated protocol identifier. It uses the same identifier as gossipsub and works with all pubsub based protocols. This allows nodes that are oblivious to Tor Push to process messages received via Tor Push.
Background # Without extensions, libp2p gossipsub does not protect sender identities.
A possible design of an anonymity extension to gossipsub is pushing messages through an anonymization network before they enter the gossipsub network. Tor is currently the largest anonymization network. It is well researched and works reliably. Basing our solution on Tor both inherits existing security research, as well as allows for a quick deployment.
Using the anonymization network approach, even the first gossipsub node that relays a given message cannot link the message to its sender (within a relatively strong adversarial model). Taking the low bandwidth overhead and the low latency overhead into consideration, Tor offers very good anonymity properties.
Functional Operation # Tor Push allows nodes to push messages over Tor into the gossipsub network. The approach specified in this document is fully backwards compatible. Gossipsub nodes that do not support Tor Push can receive and relay Tor Push messages, because Tor Push uses the same Protocol ID as gossipsub.
Messages are sent over Tor via SOCKS5. Tor Push uses a dedicated libp2p context to prevent information leakage. To significantly increase resilience and mitigate circuit failures, Tor Push establishes several connections, each to a different randomly selected gossipsub node.
Specification # This section specifies the format of Tor Push messages, as well as how Tor Push messages are received and sent, respectively.
Wire Format # The wire format of a Tor Push message corresponds verbatim to a typical libp2p pubsub message.
message Message { optional string from = 1; optional bytes data = 2; optional bytes seqno = 3; required string topic = 4; optional bytes signature = 5; optional bytes key = 6; } Receiving Tor Push Messages # Any node supporting a protocol with ID /meshsub/1.1.0 (e.g. gossipsub), can receive Tor Push messages. Receiving nodes are oblivious to Tor Push and will process incoming messages according to the respective meshsub/1.1.0 specification.
Sending Tor Push Messages # In the following, we refer to nodes sending Tor Push messages as Tp-nodes (Tor Push nodes).
Tp-nodes MUST setup a separate libp2p context, i.e. libp2p switch, which MUST NOT be used for any purpose other than Tor Push. We refer to this context as Tp-context. The Tp-context MUST NOT share any data, e.g. peer lists, with the default context.
Tp-peers are peers a Tp-node plans to send Tp-messages to. Tp-peers MUST support /meshsub/1.1.0. For retrieving Tp-peers, Tp-nodes SHOULD use an ambient peer discovery method that retrieves a random peer sample (from the set of all peers), e.g. 33/WAKU2-DISCV5.
Tp-nodes MUST establish a connection as described in sub-section Tor Push Connection Establishment to at least one Tp-peer. To significantly increase resilience, Tp-nodes SHOULD establish Tp-connections to D peers, where D is the desired gossipsub out-degree, with a default value of 8.
Each Tp-message MUST be sent via the Tp-context over at least one Tp-connection. To increase resilience, Tp-messages SHOULD be sent via the Tp-context over all available Tp-connections.
Control messages of any kind, e.g. gossipsub graft, MUST NOT be sent via Tor Push.
Connection Establishment # Tp-nodes establish a /meshsub/1.1.0 connection to tp-peers via SOCKS5 over Tor.
Establishing connections, which in turn establishes the respective Tor circuits, can be done ahead of time.
Epochs # Tor Push introduces epochs. The default epoch duration is 10 minutes. (We might adjust this default value based on experiments and evaluation in future versions of this document. It seems a good trade-off between traceablity and circuit building overhead.)
For each epoch, the Tp-context SHOULD be refreshed, which includes
libp2p peer-ID Tp-peer list connections to Tp-peers Both Tp-peer selection for the next epoch and establishing connections to the newly selected peers SHOULD be done during the current epoch and be completed before the new epoch starts. This avoids adding latency to message transmission.
Security/Privacy Considerations # Fingerprinting Attacks # Protocols that feature distinct patterns are prone to fingerprinting attacks when using them over Tor Push. Both malicious guards and exit nodes could detect these patterns and link the sender and receiver, respectively, to transmitted traffic. As a mitigation, such protocols can introduce dummy messages and/or padding to hide patterns.
DoS # General DoS against Tor # Using untargeted DoS to prevent Tor Push messages from entering the gossipsub network would cost vast resources, because Tor Push transmits messages over several circuits and the Tor network is well established.
Targeting the Guard # Denying the service of a specific guard node blocks Tp-nodes using the respective guard. Tor guard selection will replace this guard [TODO elaborate]. Still, messages might be delayed during this window which might be critical to certain applications.
Targeting the Gossipsub Network # Without sophisticated rate limiting (for example using 17/WAKU2-RLN-RELAY), attackers can spam the gossipsub network. It is not enough to just block peers that send too many messages, because these messages might actually come from a Tor exit node that many honest Tp-nodes use. Without Tor Push, protocols on top of gossipsub could block peers if they exceed a certain message rate. With Tor Push, this would allow the reputation-based DoS attack described in Bitcoin over Tor isn\u0026rsquo;t a Good Idea.
Peer Discovery # The discovery mechanism could be abused to link requesting nodes to their Tor connections to discovered nodes. An attacker that controls both the node that responds to a discovery query, and the node who’s ENR the response contains, can link the requester to a Tor connection that is expected to be opened to the node represented by the returned ENR soon after.
Further, the discovery mechanism (e.g. discv5) could be abused to distribute disproportionately many malicious nodes. For instance if p% of the nodes in the network are malicious, an attacker could manipulate the discovery to return malicious nodes with 2p% probability. The discovery mechanism needs to be resilient against this attack.
Roll-out Phase # During the roll-out phase of Tor Push, during which only a few nodes use Tor Push, attackers can narrow down the senders of Tor messages to the set of gossipsub nodes that do not originate messages. Nodes who want anonymity guarantees even during the roll-out phase can use separate network interfaces for their default context and Tp-context, respectively. For the best protection, these contexts should run on separate physical machines.
Copyright # Copyright and related rights waived via CC0.
References # libp2p gossipsub libp2p pubsub libp2p pubsub message libp2p switch SOCKS5 Tor 33/WAKU2-DISCV5 Bitcoin over Tor isn\u0026rsquo;t a Good Idea 17/WAKU2-RLN-RELAY `}),e.add({id:38,href:"/spec/47/",title:"47/WAKU2-TOR-PUSH",section:"Docs",content:` Abstract # This document extends the 11/WAKU2-RELAY, specifying Waku Tor Push, which allows nodes to push messages via Tor into the Waku relay network.
Waku Tor Push builds on 46/GOSSIPSUB-TOR-PUSH.
Protocol identifier: /vac/waku/relay/2.0.0
Note: Waku Tor Push does not have a dedicated protocol identifier. It uses the same identifier as Waku relay. This allows Waku relay nodes that are oblivious to Tor Push to process messages received via Tor Push.
Functional Operation # In its current version, Waku Tor Push corresponds to 46/GOSSIPSUB-TOR-PUSH applied to 11/WAKU2-RELAY, instead of libp2p gossipsub.
Security/Privacy Considerations # see 46/GOSSIPSUB-TOR-PUSH
Copyright # Copyright and related rights waived via CC0.
References # 11/WAKU2-RELAY libp2p gossipsub 46/GOSSIPSUB-TOR-PUSH Tor `}),e.add({id:39,href:"/spec/48/",title:"48/RLN-INTEREP-SPEC",section:"Docs",content:` Abstract # This spec integrates Interep into the RLN spec. Interep is a group management protocol that allows for the creation of groups of users and the management of their membership. It is used to manage the membership of the RLN group.
Interep ties in web2 identities with reputation, and sorts the users into groups based on their reputation score. For example, a GitHub user with over 100 followers is considered to have \u0026ldquo;gold\u0026rdquo; reputation.
Interep uses Semaphore under the hood to allow anonymous signaling of membership in a group. Therefore, a user with a \u0026ldquo;gold\u0026rdquo; reputation can prove the existence of their membership without revealing their identity.
RLN is used for spam prevention, and Interep is used for group management.
By using Interep with RLN, we allow users to join RLN membership groups without the need for on-chain financial stake.
Motivation # To have Sybil-Resistant group management, there are implementations of RLN which make use of financial stake on-chain. However, this is not ideal because it reduces the barrier of entry for honest participants.
In this case, honest participants will most likely have a web2 identity accessible to them, which can be used for joining an Interep reputation group. By modifying the RLN spec to use Interep, we can have Sybil-Resistant group management without the need for on-chain financial stake.
Since RLN and Interep both use Semaphore-style credentials, it is possible to use the same set of credentials for both.
Functional Operation # Using Interep with RLN involves the following steps -
Generate Semaphore credentials Verify reputation and join Interep group Join RLN membership group via interaction with Smart Contract, by passing a proof of membership to the Interep group 1. Generate Semaphore credentials # Semaphore credentials are generated in a standard way, depicted in the Semaphore documentation.
2. Verify reputation and join Interep group # Using the Interep app deployed on Goerli, the user can check their reputation tier and join the corresponding group. This results in a transaction to the Interep contract, which adds them to the group.
3. Join RLN membership group # Instead of sending funds to the RLN contract to join the membership group, the user can send a proof of membership to the Interep group. This proof is generated by the user, and is verified by the contract. The contract ensures that the user is a member of the Interep group, and then adds them to the RLN membership group.
Following is the modified signature of the register function in the RLN contract -
/// @param groupId: Id of the group. /// @param signal: Semaphore signal. /// @param nullifierHash: Nullifier hash. /// @param externalNullifier: External nullifier. /// @param proof: Zero-knowledge proof. /// @param idCommitment: ID Commitment of the member. function register( uint256 groupId, bytes32 signal, uint256 nullifierHash, uint256 externalNullifier, uint256[8] calldata proof, uint256 idCommitment ) Verification of messages # Messages are verified the same way as in the RLN spec.
Slashing # The slashing mechanism is the same as in the RLN spec. It is important to note that the slashing may not have the intended effect on the user, since the only consequence is that they cannot send messages. This is due to the fact that the user can send a identity commitment in the registration to the RLN contract, which is different than the one used in the Interep group.
Proof of Concept # A proof of concept is available at vacp2p/rln-interp-contract which integrates Interep with RLN.
Security Considerations # As mentioned in Slashing, the slashing mechanism may not have the intended effect on the user. This spec inherits the security considerations of the RLN spec. This spec inherits the security considerations of Interep. A user may make multiple registrations using the same Interep proofs but different identity commitments. The way to mitigate this is to check if the nullifier hash has been detected previously in proof verification. References # RLN spec Interep Semaphore Decentralized cloudflare using Interep Interep contracts RLN contract RLNP2P `}),e.add({id:40,href:"/spec/5/",title:"5/WAKU0",section:"Docs",content:`This specification describes the format of Waku messages within the ÐΞVp2p Wire Protocol. This spec substitutes EIP-627. Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic messages (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
Motivation # Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku messages in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
Definitions # Term Definition Light node A Waku node that does not forward any messages. Envelope Messages sent and received by Waku nodes. Node Some process that is able to communicate for Waku. Underlying Transports and Prerequisites # Use of DevP2P # For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
Gossip based routing # In Whisper, messages are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Messages are eligible for retransmission until their TTL expires. A node SHOULD relay messages to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a mailserver response. A node SHOULD NOT send a message to a peer that it has already sent before.
Wire Specification # Use of RLPx transport protocol # All Waku messages are sent as devp2p RLPx transport protocol, version 51 packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See informal RLP spec and the Ethereum Yellow Paper, appendix B for more details on RLP.
Waku is a RLPx subprotocol called waku with version 0. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of waku, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
ABNF specification # Using Augmented Backus-Naur form (ABNF) we have the following format:
; Packet codes 0 - 127 are reserved for Waku protocol packet-code = 1*3DIGIT ; rate limits limit-ip = 1*DIGIT limit-peerid = 1*DIGIT limit-topic = 1*DIGIT rate-limits = \u0026#34;[\u0026#34; limit-ip limit-peerid limit-topic \u0026#34;]\u0026#34; pow-requirement-key = 48 bloom-filter-key = 49 light-node-key = 50 confirmations-enabled-key = 51 rate-limits-key = 52 topic-interest-key = 53 status-options = \u0026#34;[\u0026#34; [ pow-requirement-key pow-requirement ] [ bloom-filter-key bloom-filter ] [ light-node-key light-node ] [ confirmations-enabled-key confirmations-enabled ] [ rate-limits-key rate-limits ] [ topic-interest-key topic-interest ] \u0026#34;]\u0026#34; status = \u0026#34;[\u0026#34; version status-options \u0026#34;]\u0026#34; status-update = status-options ; version is \u0026#34;an integer (as specified in RLP)\u0026#34; version = DIGIT confirmations-enabled = BIT light-node = BIT ; pow is \u0026#34;a single floating point value of PoW. ; This value is the IEEE 754 binary representation ; of a 64-bit floating point number. ; Values of qNAN, sNAN, INF and -INF are not allowed. ; Negative values are also not allowed.\u0026#34; pow = 1*DIGIT \u0026#34;.\u0026#34; 1*DIGIT pow-requirement = pow ; bloom filter is \u0026#34;a byte array\u0026#34; bloom-filter = *OCTET waku-envelope = \u0026#34;[\u0026#34; expiry ttl topic data nonce \u0026#34;]\u0026#34; ; List of topics interested in topic-interest = \u0026#34;[\u0026#34; *10000topic \u0026#34;]\u0026#34; ; 4 bytes (UNIX time in seconds) expiry = 4OCTET ; 4 bytes (time-to-live in seconds) ttl = 4OCTET ; 4 bytes of arbitrary data topic = 4OCTET ; byte array of arbitrary size ; (contains encrypted message) data = OCTET ; 8 bytes of arbitrary data ; (used for PoW calculation) nonce = 8OCTET messages = 1*waku-envelope ; mail server / client specific p2p-request = waku-envelope p2p-message = 1*waku-envelope ; packet-format needs to be paired with its ; corresponding packet-format packet-format = \u0026#34;[\u0026#34; packet-code packet-format \u0026#34;]\u0026#34; required-packet = 0 status / 1 messages / 22 status-update / optional-packet = 126 p2p-request / 127 p2p-message packet = \u0026#34;[\u0026#34; required-packet [ optional-packet ] \u0026#34;]\u0026#34; All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from 0x00.
Packet Codes # The message codes reserved for Waku protocol: 0 - 127.
Messages with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
The Waku sub-protocol MUST support the following packet codes:
Name Int Value Status 0 Messages 1 Status Update 22 The following message codes are optional, but they are reserved for specific purpose.
Name Int Value Comment Batch Ack 11 Message Response 12 P2P Request 126 P2P Message 127 Packet usage # Status # The Status message serves as a Waku handshake and peers MUST exchange this message upon connection. It MUST be sent after the RLPx handshake and prior to any other Waku messages.
A Waku node MUST await the Status message from a peer before engaging in other Waku protocol activity with that peer. When a node does not receive the Status message from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
Upon retrieval of the Status message, the node SHOULD validate the message received and validated the Status message. Note that its peer might not be in the same state.
When a node is receiving other Waku messages from a peer before a Status message is received, the node MUST ignore these messages and SHOULD disconnect from that peer. Status messages received after the handshake is completed MUST also be ignored.
The status message MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
Messages # This packet is used for sending the standard Waku envelopes.
Status Update # The Status Update message is used to communicate an update of the settings of the node. The format is the same as the Status message, all fields are optional. If none of the options are specified the message MUST be ignored and considered a noop. Fields that are omitted are considered unchanged, fields that haven\u0026rsquo;t changed SHOULD not be transmitted.
PoW Requirement update
When PoW is updated, peers MUST NOT deliver the sender envelopes with PoW lower than specified in this message.
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by message size and TTL:
PoW = (2**BestBit) / (size * TTL) PoW calculation:
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field. fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce) fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl) where size is the size of the RLP-encoded envelope, excluding env_nonce field (size of short_rlp(envelope)).
Bloom filter update
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
LET D[*] = 0 FOREACH i IN { 0, 1, 2 } DO LET n = S[i] IF S[3] \u0026amp; (2 ** i) THEN n += 256 D[n] = 1 END FOR A full bloom filter (all the bits set to 1) means that the node is to be considered a Full Node and it will accept any topic.
If both Topic Interest and bloom filter are specified, Topic Interest always takes precedence and bloom filter MUST be ignored.
If only bloom filter is specified, the current Topic Interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
Topic Interest update
This packet is used by Waku nodes for sharing their interest in messages with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
If only Topic Interest is specified, the current bloom filter MUST be discarded and only the updated Topic Interest MUST be used when forwarding or posting envelopes.
An empty array signals that the node is not currently interested in receiving any envelope.
Rate Limits update
This packet is used for informing other nodes of their self defined rate limits.
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
If a peer exceeds node\u0026rsquo;s rate limits, the connection between them MAY be dropped.
Each node SHOULD broadcast its rate limits to its peers using the rate limits packet. The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer\u0026rsquo;s rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
Message Confirmations update
Message confirmations tell a node that a message originating from it has been received by its peers, allowing a node to know whether a message has or has not been received.
A node MAY send a message confirmation for any batch of messages received with a packet Messages Code.
A message confirmation is sent using Batch Acknowledge packet or Message Response packet. The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data.
The current version of the message response is 1.
Using Augmented Backus-Naur form (ABNF) we have the following format:
; a version of the Message Response version = 1*DIGIT ; keccak256 hash of the envelopes batch data (raw bytes) for which the confirmation is sent hash = *OCTET hasherror = *OCTET ; error code code = 1*DIGIT ; a descriptive error message description = *ALPHA error = \u0026#34;[\u0026#34; hasherror code description \u0026#34;]\u0026#34; errors = *error response = \u0026#34;[\u0026#34; hash errors \u0026#34;]\u0026#34; confirmation = \u0026#34;[\u0026#34; version response \u0026#34;]\u0026#34; The supported codes: 1: means time sync error which happens when an envelope is too old or created in the future (the root cause is no time sync between nodes).
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message, a corresponding confirmation is broadcast by one or more peers.
P2P Request # This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting old messages from the Waku Mail Server.
P2P Message # This packet is used for sending the peer-to-peer messages, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of old (expired) messages, which is otherwise not allowed.
Payload Encryption # Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
Packet code Rationale # Packet codes 0x00 and 0x01 are already used in all Waku / Whisper versions. Packet code 0x02 and 0x03 were previously used in Whisper but are deprecated as of Waku v0.4
Packet code 0x22 is used to dynamically change the settings of a node.
Packet codes 0x7E and 0x7F may be used to implement Waku Mail Server and Client. Without P2P messages it would be impossible to deliver the old messages, since they will be recognized as expired, and the peer will be disconnected for violating the Whisper protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
Additional capabilities # Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
Additionally there is the capability of a mailserver which is documented in its on specification.
Light node # The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
Light nodes MUST NOT forward any incoming messages, they MUST only send their own messages. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in messages being dropped between the two.
Light nodes are identified by the light_node value in the status message.
Accounting for resources (experimental) # Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm\u0026rsquo;s SWAP protocol, and works by doing pairwise accounting for resources.
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
peer sent received peer1 0 123 peer2 10 40 In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
Upgradability and Compatibility # General principles and policy # These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
Waku aims to be compatible with previous and future versions.
In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
To achieve this, we employ the following two general strategies:
a) Accretion (including protocol negotiation) over changing data b) When we want to change things, we give it a new name (for example, a version number). Examples:
We enable bridging between shh/6 and waku/0 until such a time as when we are ready to gracefully drop support for shh/6 (1, 2, 3). When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3). To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3) When we we want to provide a new set of messages that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3) Backwards Compatibility # Waku is a different subprotocol from Whisper so it isn\u0026rsquo;t directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
Waku-Whisper bridging # waku/0 and shh/6 are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
Roles:
Waku client A, only Waku capability Whisper client B, only Whisper capability WakuWhisper bridge C, both Waku and Whisper capability Flow:
A posts message; B posts message. C picks up message from A and B and relays them both to Waku and Whisper. A receives message on Waku; B on Whisper. Note: This flow means if another bridge C1 is active, we might get duplicate relaying for a message between C1 and C2. I.e. Whisper(\u0026lt;\u0026gt;Waku\u0026lt;\u0026gt;Whisper)\u0026lt;\u0026gt;Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
Forward Compatibility # It is desirable to have a strategy for maintaining forward compatibility between waku/0 and future version of waku. Here we outline some concerns and strategy for this.
Connecting to nodes with multiple versions: The way this SHOULD be accomplished in the future is by negotiating the versions of subprotocols, within the hello message nodes transmit their capabilities along with a version. As suggested in EIP-8, if a node connects that has a higher version number for a specific capability, the node with a lower number SHOULD assume backwards compatibility. The node with the higher version will decide if compatibility can be assured between versions, if this is not the case it MUST disconnect. Adding new packet codes: New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet. Adding new options in status-options: New options can be added to the status-options association list in the status and status-update packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving status-options with unknown option keys. Appendix A: Security considerations # There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as mailservers can be found in their respective specifications.
Scalability and UX # Bandwidth usage:
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described here.
Gossip-based routing:
Use of gossip-based routing doesn\u0026rsquo;t necessarily scale. It means each node can see a message multiple times, and having too many light nodes can cause propagation probability that is too low. See Whisper vs PSS for more and a possible Kademlia based alternative.
Lack of incentives:
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
Privacy # Light node privacy:
The main privacy concern with light nodes is that directly connected peers will know that a message originates from them (as it are the only ones it sends). This means nodes can make assumptions about what messages (topics) their peers are interested in.
Bloom filter privacy:
By having a bloom filter where only the topics you are interested in are set, you reveal which messages you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the Anonymity trilemma.
Privacy guarantees not rigorous:
Privacy for Whisper / Waku haven\u0026rsquo;t been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
Topic hygiene:
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
Spam resistance # PoW bad for heterogeneous devices:
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
Censorship resistance # Devp2p TCP port blockable:
By default Devp2p runs on port 30303, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port 80 or 443, but there are still outstanding issues. See libp2p and Tor\u0026rsquo;s Pluggable Transport for how this can be improved.
Appendix B: Implementation Notes # Implementation Matrix # Client Spec supported Details Status-go 0.5 details Nimbus 0.4 details Recommendations for clients # Notes useful for implementing Waku mode.
Avoid duplicate envelopes
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
Topic specific recommendations
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
Node discovery # Resource restricted devices SHOULD use EIP-1459 to discover nodes.
Known static nodes MAY also be used.
Changelog # Version 0.6 # Released April 21,2020
Mark spec as Deprecated mode in terms of its lifecycle. Version 0.5 # Released March 17,2020
Clarify the preferred way of handling unknown keys in the status-options association list. Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior Version 0.4 # Released February 21, 2020.
Simplify implementation matrix with latest state Introduces a new required packet code Status Code (0x22) for communicating option changes Deprecates the following packet codes: PoW Requirement (0x02), Bloom Filter (0x03), Rate limits (0x20), Topic interest (0x21) - all superseded by the new Status Code (0x22) Increased topic-interest capacity from 1000 to 10000 Version 0.3 # Released February 13, 2020.
Recommend DNS based node discovery over other Discovery methods. Mark spec as Draft mode in terms of its lifecycle. Simplify Changelog and misc formatting. Handshake/Status message not compatible with shh/6 nodes; specifying options as association list. Include topic-interest in Status handshake. Upgradability policy. topic-interest packet code. Version 0.2 # Released December 10, 2019.
General style improvements. Fix ABNF grammar. Mailserver requesting/receiving. New packet codes: topic-interest (experimental), rate limits (experimental). More details on handshake modifications. Accounting for resources mode (experimental) Appendix with security considerations: scalability and UX, privacy, and spam resistance. Appendix with implementation notes and implementation matrix across various clients with breakdown per capability. More details on handshake and parameters. Describe rate limits in more detail. More details on mailserver and mail client API. Accounting for resources mode (very experimental). Clarify differences with Whisper. Version 0.1 # Initial version. Released November 21, 2019.
Differences between shh/6 and waku/0 # Summary of main differences between this spec and Whisper v6, as described in EIP-627:
RLPx subprotocol is changed from shh/6 to waku/0. Light node capability is added. Optional rate limiting is added. Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits Mail Server and Mail Client functionality is now part of the specification. P2P Message packet contains a list of envelopes instead of a single envelope. Copyright # Copyright and related rights waived via CC0.
Footnotes # Felix Lange et al. The RLPx Transport Protocol. Ethereum.\u0026#160;\u0026#x21a9;\u0026#xfe0e;
`}),e.add({id:41,href:"/spec/51/",title:"51/WAKU2-RELAY-SHARDING",section:"Docs",content:` Abstract # This document describes ways of sharding the Waku relay topic, allowing Waku networks to scale in the number of content topics.
Note: Scaling in the size of a single content topic is out of scope for this document.
Background and Motivation # Unstructured P2P networks are more robust and resilient against DoS attacks compared to structured P2P networks). However, they do not scale to large traffic loads. A single libp2p gossipsub mesh, which carries messages associated with a single pubsub topic, can be seen as a separate unstructured P2P network (control messages go beyond these boundaries, but at its core, it is a separate P2P network). With this, the number of Waku relay content topics that can be carried over a pubsub topic is limited. This prevents app protocols that aim to span many multicast groups (realized by content topics) from scaling.
This document specifies three pubsub topic sharding methods (with varying degrees of automation), which allow application protocols to scale in the number of content topics. This document also covers discovery of topic shards.
Named Sharding # Named sharding offers apps to freely choose pubsub topic names. It is RECOMMENDED for App protocols to follow the naming structure detailed in 23/WAKU2-TOPICS. With named sharding, managing discovery falls into the responsibility of apps.
From an app protocol point of view, a subscription to a content topic waku2/xxx on a shard named /mesh/v1.1.1/xxx would look like:
subscribe(\u0026quot;/waku2/xxx\u0026quot;, \u0026quot;/mesh/v1.1.1/xxx\u0026quot;)
Static Sharding # Static sharding offers a set of shards with fixed names. Assigning content topics to specific shards is up to app protocols, but the discovery of these shards is managed by Waku.
Static shards are managed in shard clusters of 1024 shards per cluster. Waku static sharding can manage $2^16$ shard clusters. Each shard cluster is identified by its index (between $0$ and $2^16-1$).
A specific shard cluster is either globally available to all apps, specific for an app protocol, or reserved for automatic sharding (see next section).
Note: This leads to $2^16 * 1024 = 2^26$ shards for which Waku manages discovery.
App protocols can either choose to use global shards, or app specific shards.
Like the IANA ports, shard clusters are divided into ranges:
index (range) usage 0 - 15 reserved 16 - 65535 app-defined networks The informational RFC 52/WAKU2-RELAY-STATIC-SHARD-ALLOC lists the current index allocations.
The global shard with index 0 and the \u0026ldquo;all app protocols\u0026rdquo; range are treated in the same way, but choosing shards in the global cluster has a higher probability of sharing the shard with other apps. This offers k-anonymity and better connectivity, but comes at a higher bandwidth cost.
The name of the pubsub topic corresponding to a given static shard is specified as
/waku/2/rs/\u0026lt;cluster_id\u0026gt;/\u0026lt;shard_number\u0026gt;,
an example for the 2nd shard in the global shard cluster:
/waku/2/rs/0/2.
Note: Because all shards distribute payload defined in 14/WAKU2-MESSAGE via protocol buffers, the pubsub topic name does not explicitly add /proto to indicate protocol buffer encoding. We use rs to indicate these are relay shard clusters; further shard types might follow in the future.
From an app point of view, a subscription to a content topic waku2/xxx on a static shard would look like:
subscribe(\u0026quot;/waku2/xxx\u0026quot;, 43)
for global shard 43. And for shard 43 of the Status app (which has allocated index 16):
subscribe(\u0026quot;/waku2/xxx\u0026quot;, 16, 43)
Discovery # Waku v2 supports the discovery of peers within static shards, so app protocols do not have to implement their own discovery method.
Nodes add information about their shard participation in their 31/WAKU2-ENR. Having a static shard participation indication as part of the ENR allows nodes to discover peers that are part of shards via 33/WAKU2-DISCV5 as well as via DNS.
Note: In the current version of this document, sharding information is directly added to the ENR. (see Ethereum ENR sharding bit vector here Static relay sharding supports 1024 shards per cluster, leading to a flag field of 128 bytes. This already takes half (including index and key) of the ENR space of 300 bytes. For this reason, the current specification only supports a single shard cluster per node. In future versions, we will add further (hierarchical) discovery methods. We will update 31/WAKU2-ENR accordingly, once this RFC moves forward.
This document specifies two ways of indicating shard cluster participation. The index list SHOULD be used for nodes that participante in fewer than 64 shards, the bit vector representation SHOULD be used for nodes participating in 64 or more shards. Nodes MUST NOT use both index list (rs) and bit vector (rsv) in a single ENR. ENRs with both rs and rsv keys SHOULD be ignored. Nodes MAY interpret rs in such ENRs, but MUST ignore rsv.
Index List # key value rs \u0026lt;2-byte shard cluster index\u0026gt; | \u0026lt;1-byte length\u0026gt; | \u0026lt;2-byte shard index\u0026gt; | \u0026hellip; | \u0026lt;2-byte shard index\u0026gt; The ENR key is rs. The value is comprised of
a two-byte shard cluster index in network byte order, concatenated with a one-byte length field holding the number of shards in the given shard cluster, concatenated with two-byte shard indices in network byte order Example:
key value rs 16u16 | 3u8 | 13u16 | 14u16 | 45u16 This example node is part of shards 13, 14, and 45 in the Status main-net shard cluster (index 16).
Bit Vector # key value rsv \u0026lt;2-byte shard cluster index\u0026gt; | \u0026lt;128-byte flag field\u0026gt; The ENR key is rsv. The value is comprised of a two-byte shard cluster index in network byte order concatenated with a 128-byte wide bit vector. The bit vector indicates which shards of the respective shard cluster the node is part of. The right-most bit in the bit vector represents shard 0, the left-most bit represents shard 1023. The representation in the ENR is inspired by Ethereum shard ENRs), and this).
Example:
key value rsv 16u16 | 0x[...]0000100000003000 The [...] in the example indicates 120 0 bytes. This example node is part of shards 13, 14, and 45 in the Status main-net shard cluster (index 16). (This is just for illustration purposes, a node that is only part of three shards should use the index list method specified above.)
Automatic Sharding # Autosharding selects shards automatically and is the default behavior for shard choice. The other choices being static and named sharding as seen in previous sections. Shards (pubsub topics) SHOULD be computed from content topics with the procedure below.
Algorithm # Hash using Sha2-256 the concatenation of the content topic application field (UTF-8 string of N bytes) and the version (UTF-8 string of N bytes). The shard to use is the modulo of the hash by the number of shards in the network.
Example # Field Value Hex application \u0026ldquo;myapp\u0026rdquo; 0x6d79617070 version \u0026ldquo;1\u0026rdquo; 0x31 network shards 8 0x8 SHA2-256 of 0x6d7961707031 is 0x8e541178adbd8126068c47be6a221d77d64837221893a8e4e53139fb802d4928 0x8e541178adbd8126068c47be6a221d77d64837221893a8e4e53139fb802d4928 MOD 8 equals 0 The shard to use has index 0 Content Topics Format for Autosharding # Content topics MUST follow the format in 23/WAKU2-TOPICS. In addition, a generation prefix MAY be added to content topics. When omitted default values are used. Generation default value is 0.
The full length format is /{generation}/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding} The short length format is /{application-name}/{version-of-the-application}/{content-topic-name}/{encoding} Example # Full length /0/myapp/1/mytopic/cbor Short length /myapp/1/mytopic/cbor Generation # The generation number monotonously increases and indirectly refers to the total number of shards of the Waku Network.
Topic Design # Content topics have 2 purposes: filtering and routing. Filtering is done by changing the {content-topic-name} field. As this part is not hashed, it will not affect routing (shard selection). The {application-name} and {version-of-the-application} fields do affect routing. Using multiple content topics with different {application-name} field has advantages and disadvantages. It increases the traffic a relay node is subjected to when subscribed to all topics. It also allows relay and light nodes to subscribe to a subset of all topics.
Problems # Hot Spots # Hot spots occur (similar to DHTs), when a specific mesh network (shard) becomes responsible for (several) large multicast groups (content topics). The opposite problem occurs when a mesh only carries multicast groups with very few participants: this might cause bad connectivity within the mesh.
The current autosharding method does not solve this problem.
Note: Automatic sharding based on network traffic measurements to avoid hot spots in not part of this specification.
Discovery # For the discovery of automatic shards this document specifies two methods (the second method will be detailed in a future version of this document).
The first method uses the discovery introduced above in the context of static shards.
The second discovery method will be a successor to the first method, but is planned to preserve the index range allocation. Instead of adding the data to the ENR, it will treat each array index as a capability, which can be hierarchical, having each shard in the indexed shard cluster as a sub-capability. When scaling to a very large number of shards, this will avoid blowing up the ENR size, and allows efficient discovery. We currently use 33/WAKU2-DISCV5 for discovery, which is based on Ethereum\u0026rsquo;s discv5. While this allows to sample nodes from a distributed set of nodes efficiently and offers good resilience, it does not allow to efficiently discover nodes with specific capabilities within this node set. Our research log post explains this in more detail. Adding efficient (but still preserving resilience) capability discovery to discv5 is ongoing research. A paper on this has been completed, but the Ethereum discv5 specification has yet to be updated. When the new capability discovery is available, this document will be updated with a specification of the second discovery method. The transition to the second method will be seamless and fully backwards compatible because nodes can still advertise and discover shard memberships in ENRs.
Security/Privacy Considerations # See 45/WAKU2-ADVERSARIAL-MODELS, especially the parts on k-anonymity. We will add more on security considerations in future versions of this document.
Receiver Anonymity # The strength of receiver anonymity, i.e. topic receiver unlinkablity, depends on the number of content topics (k), as a proxy for the number of peers and messages, that get mapped onto a single pubsub topic (shard). For named and static sharding this responsibility is at the app protocol layer.
Copyright # Copyright and related rights waived via CC0.
References # 11/WAKU2-RELAY Unstructured P2P network 33/WAKU2-DISCV5 31/WAKU2-ENR 23/WAKU2-TOPICS 51/WAKU2-RELAY-SHARDING Ethereum ENR sharding bit vector Ethereum discv5 specification Research log: Waku Discovery 45/WAKU2-ADVERSARIAL-MODELS `}),e.add({id:42,href:"/spec/52/",title:"52/WAKU2-RELAY-STATIC-SHARD-ALLOC",section:"Docs",content:` Abstract # This document lists static shard flag index assignments (see 51/WAKU2-RELAY-SHARDING.
Background # Similar to the IANA port allocation, this document lists static shard index assignments (see 51/WAKU2-RELAY-SHARDING.
Assingment Process # Note: Future versions of this document will specify the assignment process.
List of Cluster Ids # index Protocol/App Description 0 global global use 1 reserved The Waku Network 2 reserved 3 reserved 4 reserved 5 reserved 6 reserved 7 reserved 8 reserved 9 reserved 10 reserved 11 reserved 12 reserved 13 reserved 14 reserved 15 reserved 16 Status Status main net 17 Status 18 Status Copyright # Copyright and related rights waived via CC0.
References # 51/WAKU2-RELAY-SHARDING IANA port allocation `}),e.add({id:43,href:"/spec/53/",title:"53/WAKU2-X3DH",section:"Docs",content:` Abstract # This document describes a method that can be used to provide a secure channel between two peers, and thus provide confidentiality, integrity, authenticity and forward secrecy. It is transport-agnostic and works over asynchronous networks.
It builds on the X3DH and Double Ratchet specifications, with some adaptations to operate in a decentralized environment.
Motivation # Nodes on a network may want to communicate with each other in a secure manner, without other nodes network being able to read their messages.
Specification # Definitions # Perfect Forward Secrecy is a feature of specific key-agreement protocols which provide assurances that session keys will not be compromised even if the private keys of the participants are compromised. Specifically, past messages cannot be decrypted by a third-party who manages to get a hold of a private key.
Secret channel describes a communication channel where a Double Ratchet algorithm is in use.
Design Requirements # Confidentiality: The adversary should not be able to learn what data is being exchanged between two Status clients. Authenticity: The adversary should not be able to cause either endpoint to accept data from any third party as though it came from the other endpoint. Forward Secrecy: The adversary should not be able to learn what data was exchanged between two clients if, at some later time, the adversary compromises one or both of the endpoints. Integrity: The adversary should not be able to cause either endpoint to accept data that has been tampered with. All of these properties are ensured by the use of Signal\u0026rsquo;s Double Ratchet
Conventions # Types used in this specification are defined using the Protobuf wire format.
Specification # End-to-End Encryption # End-to-end encryption (E2EE) takes place between two clients. The main cryptographic protocol is a Double Ratchet protocol, which is derived from the Off-the-Record protocol, using a different ratchet. The Waku v2 protocol subsequently encrypts the message payload, using symmetric key encryption. Furthermore, the concept of prekeys (through the use of X3DH) is used to allow the protocol to operate in an asynchronous environment. It is not necessary for two parties to be online at the same time to initiate an encrypted conversation.
Cryptographic Protocols # This protocol uses the following cryptographic primitives:
X3DH
Elliptic curve Diffie-Hellman key exchange (secp256k1) KECCAK-256 ECDSA ECIES Double Ratchet
HMAC-SHA-256 as MAC Elliptic curve Diffie-Hellman key exchange (Curve25519) AES-256-CTR with HMAC-SHA-256 and IV derived alongside an encryption key The node achieves key derivation using HKDF.
Pre-keys # Every client SHOULD initially generate some key material which is stored locally:
Identity keypair based on secp256k1 - IK A signed prekey based on secp256k1 - SPK A prekey signature - Sig(IK, Encode(SPK)) More details can be found in the X3DH Prekey bundle creation section of 2/ACCOUNT.
Prekey bundles MAY be extracted from any peer\u0026rsquo;s messages, or found via searching for their specific topic, {IK}-contact-code.
The following methods can be used to retrieve prekey bundles from a peer\u0026rsquo;s messages:
contact codes; public and one-to-one chats; QR codes; ENS record; Decentralized permanent storage (e.g. Swarm, IPFS). Waku Waku SHOULD be used for retrieving prekey bundles.
Since bundles stored in QR codes or ENS records cannot be updated to delete already used keys, the bundle MAY be rotated every 24 hours, and distributed via Waku.
Flow # The key exchange can be summarized as follows:
Initial key exchange: Two parties, Alice and Bob, exchange their prekey bundles, and derive a shared secret.
Double Ratchet: The two parties use the shared secret to derive a new encryption key for each message they send.
Chain key update: The two parties update their chain keys. The chain key is used to derive new encryption keys for future messages.
Message key derivation: The two parties derive a new message key from their chain key, and use it to encrypt a message.
1. Initial key exchange flow (X3DH) # Section 3 of the X3DH protocol describes the initial key exchange flow, with some additional context:
The peers\u0026rsquo; identity keys IK_A and IK_B correspond to their public keys; Since it is not possible to guarantee that a prekey will be used only once in a decentralized world, the one-time prekey OPK_B is not used in this scenario; Nodes SHOULD not send Bundles to a centralized server, but instead provide them in a decentralized way as described in the Pre-keys section. Alice retrieves Bob\u0026rsquo;s prekey bundle, however it is not specific to Alice. It contains:
(reference wire format)
Wire format:
// X3DH prekey bundle message Bundle { // Identity key \u0026#39;IK_B\u0026#39; bytes identity = 1; // Signed prekey \u0026#39;SPK_B\u0026#39; for each device, indexed by \u0026#39;installation-id\u0026#39; map\u0026lt;string,SignedPreKey\u0026gt; signed_pre_keys = 2; // Prekey signature \u0026#39;Sig(IK_B, Encode(SPK_B))\u0026#39; bytes signature = 4; // When the bundle was created locally int64 timestamp = 5; } (reference wire format)
message SignedPreKey { bytes signed_pre_key = 1; uint32 version = 2; } The signature is generated by sorting installation-id in lexicographical order, and concatenating the signed-pre-key and version:
installation-id-1signed-pre-key1version1installation-id2signed-pre-key2-version-2
2. Double Ratchet # Having established the initial shared secret SK through X3DH, it SHOULD be used to seed a Double Ratchet exchange between Alice and Bob.
Refer to the Double Ratchet spec for more details.
The initial message sent by Alice to Bob is sent as a top-level ProtocolMessage (reference wire format) containing a map of DirectMessageProtocol indexed by installation-id (reference wire format):
message ProtocolMessage { // The installation id of the sender string installation_id = 2; // A sequence of bundles repeated Bundle bundles = 3; // One to one message, encrypted, indexed by installation_id map\u0026lt;string,DirectMessageProtocol\u0026gt; direct_message = 101; // Public message, not encrypted bytes public_message = 102; } message EncryptedMessageProtocol { X3DHHeader X3DH_header = 1; DRHeader DR_header = 2; DHHeader DH_header = 101; // Encrypted payload // if a bundle is available, contains payload encrypted with the Double Ratchet algorithm; // otherwise, payload encrypted with output key of DH exchange (no Perfect Forward Secrecy). bytes payload = 3; } Where:
X3DH_header: the X3DHHeader field in DirectMessageProtocol contains:
(reference wire format)
message X3DHHeader { // Alice\u0026#39;s ephemeral key \`EK_A\` bytes key = 1; // Bob\u0026#39;s bundle signed prekey bytes id = 4; } DR_header: Double ratchet header (reference wire format). Used when Bob\u0026rsquo;s public bundle is available:
message DRHeader { // Alice\u0026#39;s current ratchet public key (as mentioned in [DR spec section 2.2](https://signal.org/docs/specifications/doubleratchet/#symmetric-key-ratchet)) bytes key = 1; // number of the message in the sending chain uint32 n = 2; // length of the previous sending chain uint32 pn = 3; // Bob\u0026#39;s bundle ID bytes id = 4; } DH_header: Diffie-Hellman header (used when Bob\u0026rsquo;s bundle is not available): (reference wire format)
message DHHeader { // Alice\u0026#39;s compressed ephemeral public key. bytes key = 1; } 3. Chain key update # The chain key MUST be updated according to the DR_Header received in the EncryptedMessageProtocol message, described in 2.Double Ratchet.
4. Message key derivation # The message key MUST be derived from a single ratchet step in the symmetric-key ratchet as described in Symmetric key ratchet
The message key MUST be used to encrypt the next message to be sent.
Security Considerations # Inherits the security considerations of X3DH and Double Ratchet.
Inherits the security considerations of the Waku v2 protocol.
The protocol is designed to be used in a decentralized manner, however, it is possible to use a centralized server to serve prekey bundles. In this case, the server is trusted.
Privacy Considerations # This protocol does not provide message unlinkability. It is possible to link messages signed by the same keypair. Copyright # Copyright and related rights waived via CC0.
References # 5/SECURE-TRANSPORT 10/WAKU2 X3DH HKDF Double Ratchet `}),e.add({id:44,href:"/spec/54/",title:"54/WAKU2-X3DH-SESSIONS",section:"Docs",content:` Abstract # This document specifies how to manage sessions based on an X3DH key exchange. This includes how to establish new sessions, how to re-establish them, how to maintain them, and how to close them.
53/WAKU2-X3DH specifies the Waku X3DH protocol for end-to-end encryption. Once two peers complete an X3DH handshake, they SHOULD establish an X3DH session.
Session Establishment # A node identifies a peer by their installation-id which MAY be interpreted as a device identifier.
Discovery of pre-key bundles # The node\u0026rsquo;s pre-key bundle MUST be broadcast on a content topic derived from the node\u0026rsquo;s public key, so that the first message may be PFS-encrypted. Each peer MUST publish their pre-key bundle periodically to this topic, otherwise they risk not being able to perform key-exchanges with other peers. Each peer MAY publish to this topic when their metadata changes, so that the other peer can update their local record.
If peer A wants to send a message to peer B, it MUST derive the topic from peer B\u0026rsquo;s public key, which has been shared out of band. Partitioned topics have been used to balance privacy and efficiency of broadcasting pre-key bundles.
The number of partitions that MUST be used is 5000.
The topic MUST be derived as follows:
var partitionsNum *big.Int = big.NewInt(5000) var partition *big.Int = big.NewInt(0).Mod(peerBPublicKey, partitionsNum) partitionTopic := \u0026#34;contact-discovery-\u0026#34; + strconv.FormatInt(partition.Int64(), 10) var hash []byte = keccak256(partitionTopic) var topicLen int = 4 if len(hash) \u0026lt; topicLen { topicLen = len(hash) } var contactCodeTopic [4]byte for i = 0; i \u0026lt; topicLen; i++ { contactCodeTopic[i] = hash[i] } Initialization # A node initializes a new session once a successful X3DH exchange has taken place. Subsequent messages will use the established session until re-keying is necessary.
Negotiated topic to be used for the session # After the peers have performed the initial key exchange, they MUST derive a topic from their shared secret to send messages on. To obtain this value, take the first four bytes of the keccak256 hash of the shared secret encoded in hexadecimal format.
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared( ecies.ImportECDSAPublic(theirPublicKey), 16, 16, ) hexEncodedKey := hex.EncodeToString(sharedKey) var hash []byte = keccak256(hexEncodedKey) var topicLen int = 4 if len(hash) \u0026lt; topicLen { topicLen = len(hash) } var topic [4]byte for i = 0; i \u0026lt; topicLen; i++ { topic[i] = hash[i] } To summarize, following is the process for peer B to establish a session with peer A:
Listen to peer B\u0026rsquo;s Contact Code Topic to retrieve their bundle information, including a list of active devices Peer A sends their pre-key bundle on peer B\u0026rsquo;s partitioned topic Peer A and peer B perform the key-exchange using the shared pre-key bundles The negotiated topic is derived from the shared secret Peers A \u0026amp; B exchange messages on the negotiated topic Concurrent sessions # If a node creates two sessions concurrently between two peers, the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
Re-keying # On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
Multi-device support # Multi-device support is quite challenging as there is not a central place where information on which and how many devices (identified by their respective installation-id) a peer has, is stored.
Furthermore, account recovery always needs to be taken into consideration, where a user wipes clean the whole device and the node loses all the information about any previous sessions. Taking these considerations into account, the way the network propagates multi-device information using X3DH bundles, which will contain information about paired devices as well as information about the sending device. This means that every time a new device is paired, the bundle needs to be updated and propagated with the new information, the user has the responsibility to make sure the pairing is successful.
The method is loosely based on Signal\u0026rsquo;s Sesame Algorithm.
Pairing # A new installation-id MUST be generated on a per-device basis. The device should be paired as soon as possible if other devices are present.
If a bundle is received, which has the same IK as the keypair present on the device, the devices MAY be paired. Once a user enables a new device, a new bundle MUST be generated which includes pairing information.
The bundle MUST be propagated to contacts through the usual channels.
Removal of paired devices is a manual step that needs to be applied on each device, and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
Sending messages to a paired group # When sending a message, the peer SHOULD send a message to other installation-id that they have seen. The node caps the number of devices to n, ordered by last activity. The node sends messages using pairwise encryption, including their own devices.
Where n is the maximum number of devices that can be paired.
Account recovery # Account recovery is the same as adding a new device, and it MUST be handled the same way.
Partitioned devices # In some cases (i.e. account recovery when no other pairing device is available, device not paired), it is possible that a device will receive a message that is not targeted to its own installation-id. In this case an empty message containing bundle information MUST be sent back, which will notify the receiving end not to include the device in any further communication.
Security Considerations # Inherits all security considerations from 53/WAKU2-X3DH. Recommendations # The value of n SHOULD be configured by the app-protocol. The default value SHOULD be 3, since a larger number of devices will result in a larger bundle size, which may not be desirable in a peer-to-peer network. Copyright # Copyright and related rights waived via CC0.
References # 53/WAKU2-X3DH Signal\u0026rsquo;s Sesame Algorithm 5/SECURE-TRANSPORT `}),e.add({id:45,href:"/spec/55/",title:"55/STATUS-1TO1-CHAT",section:"Docs",content:` Abstract # This specification describes how the Status 1-to-1 chat protocol is implemented on top of the Waku v2 protocol. This protocol can be used to send messages to a single recipient.
Terminology # Participant: A participant is a user that is able to send and receive messages. 1-to-1 chat: A chat between two participants. Public chat: A chat where any participant can join and read messages. Private chat: A chat where only invited participants can join and read messages. Group chat: A chat where multiple select participants can join and read messages. Group admin: A participant that is able to add/remove participants from a group chat. Background # This document describes how 2 peers communicate with each other to send messages in a 1-to-1 chat, with privacy and authenticity guarantees.
Specification # Overview # This protocol MAY use any key-exchange mechanism previously discussed -
53/WAKU2-X3DH 35/WAKU2-NOISE This protocol can provide end-to-end encryption to give peers a strong degree of privacy and security. Public chat messages are publicly readable by anyone since there\u0026rsquo;s no permission model for who is participating in a public chat.
Flow # Negotiation of a 1:1 chat # There are two phases in the initial negotiation of a 1:1 chat:
Identity verification (e.g., face-to-face contact exchange through QR code, Identicon matching). A QR code serves two purposes simultaneously - identity verification and initial key material retrieval; Asynchronous initial key exchange For more information on account generation and trust establishment, see 2/ACCOUNT
Post Negotiation # After the peers have shared their public key material, a 1:1 chat can be established using the methods described in the key-exchange protocols mentioned above.
Session management # The 1:1 chat is made robust by having sessions between peers. It is handled by the key-exchange protocol used. For example,
53/WAKU2-X3DH, the session management is described in 54/WAKU2-X3DH-SESSIONS
35/WAKU2-NOISE, the session management is described in 37/WAKU2-NOISE-SESSIONS
Negotiation of a 1:1 chat amongst multiple participants (group chat) # A small, private group chat can be constructed by having multiple participants negotiate a 1:1 chat amongst each other. Each participant MUST maintain a session with all other participants in the group chat. This allows for a group chat to be created with a small number of participants.
However, this method does not scale as the number of participants increases, for the following reasons -
The number of messages sent over the network increases with the number of participants. Handling the X3DH key exchange for each participant is computationally expensive. The above issues are addressed in 56/STATUS-COMMUNITIES, with other trade-offs.
Flow # The following flow describes how a group chat is created and maintained.
Membership Update Flow # Membership updates have the following wire format:
message MembershipUpdateMessage { // The chat id of the private group chat // derived in the following way: // chat_id = hex(chat_creator_public_key) + \u0026#34;-\u0026#34; + random_uuid // This chat_id MUST be validated by all participants string chat_id = 1; // A list of events for this group chat, first 65 bytes are the signature, then is a // protobuf encoded MembershipUpdateEvent repeated bytes events = 2; oneof chat_entity { // An optional chat message ChatMessage message = 3; // An optional reaction to a message EmojiReaction emoji_reaction = 4; } } Note that in events, the first element is the signature, and all other elements after are encoded MembershipUpdateEvent\u0026rsquo;s.
where MembershipUpdateEvent is defined as follows:
message MembershipUpdateEvent { // Lamport timestamp of the event uint64 clock = 1; // Optional list of public keys of the targets of the action repeated string members = 2; // Name of the chat for the CHAT_CREATED/NAME_CHANGED event types string name = 3; // The type of the event EventType type = 4; // Color of the chat for the CHAT_CREATED/COLOR_CHANGED event types string color = 5; // Chat image bytes image = 6; enum EventType { UNKNOWN = 0; CHAT_CREATED = 1; // See [CHAT_CREATED](#chat-created) NAME_CHANGED = 2; // See [NAME_CHANGED](#name-changed) MEMBERS_ADDED = 3; // See [MEMBERS_ADDED](#members-added) MEMBER_JOINED = 4; // See [MEMBER_JOINED](#member-joined) MEMBER_REMOVED = 5; // See [MEMBER_REMOVED](#member-removed) ADMINS_ADDED = 6; // See [ADMINS_ADDED](#admins-added) ADMIN_REMOVED = 7; // See [ADMIN_REMOVED](#admin-removed) COLOR_CHANGED = 8; // See [COLOR_CHANGED](#color-changed) IMAGE_CHANGED = 9; // See [IMAGE_CHANGED](#image-changed) } } Note that the definitions for ChatMessage and EmojiReaction can be found in chat_message.proto and emoji_reaction.proto.
Chat Created # When creating a group chat, this is the first event that MUST be sent. Any event with a clock value lower than this MUST be discarded. Upon receiving this event a client MUST validate the chat_id provided with the update and create a chat with identified by chat_id.
By default, the creator of the group chat is the only group admin.
Name Changed # To change the name of the group chat, group admins MUST use a NAME_CHANGED event. Upon receiving this event a client MUST validate the chat_id provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. If the event is valid the chat name SHOULD be changed according to the provided message.
Members Added # To add members to the chat, group admins MUST use a MEMBERS_ADDED event. Upon receiving this event a participant MUST validate the chat_id provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. If the event is valid, a participant MUST update the list of members of the chat who have not joined, adding the members received.
Member Joined # To signal the intent to start receiving messages from a given chat, new participants MUST use a MEMBER_JOINED event. Upon receiving this event a participant MUST validate the chat_id provided with the updates. If the event is valid a participant MUST add the new participant to the list of participants stored locally. Any message sent to the group chat MUST now include the new participant.
Member Removed # There are two ways in which a member MAY be removed from a group chat:
A member MAY leave the chat by sending a MEMBER_REMOVED event, with the members field containing their own public key. An admin MAY remove a member by sending a MEMBER_REMOVED event, with the members field containing the public key of the member to be removed. Each participant MUST validate the chat_id provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. If the event is valid, a participant MUST update the local list of members accordingly.
Admins Added # To promote participants to group admin, group admins MUST use an ADMINS_ADDED event. Upon receiving this event, a participant MUST validate the chat_id provided with the updates, MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. If the event is valid, a participant MUST update the list of admins of the chat accordingly.
Admin Removed # Group admins MUST NOT be able to remove other group admins. An admin MAY remove themselves by sending an ADMIN_REMOVED event, with the members field containing their own public key. Each participant MUST validate the chat_id provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored. If the event is valid, a participant MUST update the list of admins of the chat accordingly.
Color Changed # To change the text color of the group chat name, group admins MUST use a COLOR_CHANGED event.
Image Changed # To change the display image of the group chat, group admins MUST use an IMAGE_CHANGED event.
Security Considerations # Inherits the security considerations of the key-exchange mechanism used, e.g., 53/WAKU2-X3DH or 35/WAKU2-NOISE Copyright # Copyright and related rights waived via CC0.
References # 53/WAKU2-X3DH 35/WAKU2-NOISE 65/STATUS-ACCOUNT 54/WAKU2-X3DH-SESSIONS 37/WAKU2-NOISE-SESSIONS 56/STATUS-COMMUNITIES chat_message.proto emoji_reaction.proto `}),e.add({id:46,href:"/spec/56/",title:"56/STATUS-COMMUNITIES",section:"Docs",content:` Abstract # This document describes the design of Status Communities over Waku v2, allowing for multiple users to communicate in a group chat. This is a key feature for the Status messaging app.
Background and Motivation # The purpose of Status communities, as specified in this document, is allowing for large group chats. Communities can have further substructure, e.g. specific channels.
Smaller group chats, on the other hand, are out of scope for this document and can be built over 55/STATUS-1TO1-CHAT. We refer to these smaller group chats simply as \u0026ldquo;group chats\u0026rdquo;, to differentiate them from Communities.
For group chats based on 55/STATUS-1TO1-CHAT, the key exchange mechanism MUST be X3DH, as described in 53/WAKU2-X3DH.
However, this method does not scale as the number of participants increases, for the following reasons -
The number of messages sent over the network increases with the number of participants. Handling the X3DH key exchange for each participant is computationally expensive. Having multicast channels reduces the overhead of a group chat based on 1:1 chat. Additionally, if all the participants of the group chat have a shared key, then the number of messages sent over the network is reduced to one per message.
Terminology # Community: A group of peers that can communicate with each other. Member: A peer that is part of a community. Admin: A member that has administrative privileges. Used interchangeably with \u0026ldquo;owner\u0026rdquo;. Channel: A designated subtopic for a community. Used interchangeably with \u0026ldquo;chat\u0026rdquo;. Design Requirements # Due to the nature of communities, the following requirements are necessary for the design of communities -
The creator of the Community is the owner of the Community. The Community owner is trusted. The Community owner can add or remove members from the Community. This extends to banning and kicking members. The Community owner can add, edit and remove channels. Community members can send/receive messages to the channels which they have access to. Communities may be encrypted (private) or unencrypted (public). A Community is uniquely identified by a public key. The public key of the Community is shared out of band. The metadata of the Community can be found by listening on a content topic derived from the public key of the Community. Community members run their own Waku nodes, with the configuration described in Waku-Protocols. Light nodes solely implementing 19/WAKU2-LIGHTPUSH may not be able to run their own Waku node with the configuration described. Design # Cryptographic Primitives # The following cryptographic primitives are used in the design -
X3DH Single Ratchet The single ratchet is used to encrypt the messages sent to the Community. The single ratchet is re-keyed when a member is added/removed from the Community. Wire format # syntax = \u0026#34;proto3\u0026#34;; message IdentityImage { // payload is a context based payload for the profile image data, // context is determined by the \`source_type\` bytes payload = 1; // source_type signals the image payload source SourceType source_type = 2; // image_type signals the image type and method of parsing the payload ImageType image_type = 3; // encryption_keys is a list of encrypted keys that can be used to decrypt an encrypted payload repeated bytes encryption_keys = 4; // encrypted signals the encryption state of the payload, default is false. bool encrypted = 5; // SourceType are the predefined types of image source allowed enum SourceType { UNKNOWN_SOURCE_TYPE = 0; // RAW_PAYLOAD image byte data RAW_PAYLOAD = 1; // ENS_AVATAR uses the ENS record\u0026#39;s resolver get-text-data.avatar data // The \`payload\` field will be ignored if ENS_AVATAR is selected // The application will read and parse the ENS avatar data as image payload data, URLs will be ignored // The parent \`ChatMessageIdentity\` must have a valid \`ens_name\` set ENS_AVATAR = 2; } } // SocialLinks represents social link associated with given chat identity (personal/community) message SocialLink { // Type of the social link string text = 1; // URL of the social link string url = 2; } // ChatIdentity represents identity of a community/chat message ChatIdentity { // Lamport timestamp of the message uint64 clock = 1; // ens_name is the valid ENS name associated with the chat key string ens_name = 2; // images is a string indexed mapping of images associated with an identity map\u0026lt;string, IdentityImage\u0026gt; images = 3; // display name is the user set identity string display_name = 4; // description is the user set description string description = 5; string color = 6; string emoji = 7; repeated SocialLink social_links = 8; // first known message timestamp in seconds (valid only for community chats for now) // 0 - unknown // 1 - no messages uint32 first_message_timestamp = 9; } message Grant { // Community ID (The public key of the community) bytes community_id = 1; // The member ID (The public key of the member) bytes member_id = 2; // The chat for which the grant is given string chat_id = 3; // The Lamport timestamp of the grant uint64 clock = 4; } message CommunityMember { // The roles a community member MAY have enum Roles { UNKNOWN_ROLE = 0; ROLE_ALL = 1; ROLE_MANAGE_USERS = 2; ROLE_MODERATE_CONTENT = 3; } repeated Roles roles = 1; } message CommunityPermissions { // The type of access a community MAY have enum Access { UNKNOWN_ACCESS = 0; NO_MEMBERSHIP = 1; INVITATION_ONLY = 2; ON_REQUEST = 3; } // If the community should be available only to ens users bool ens_only = 1; // If the community is private bool private = 2; Access access = 3; } message CommunityAdminSettings { // If the Community admin may pin messages bool pin_message_all_members_enabled = 1; } message CommunityChat { // A map of members in the community to their roles in a chat map\u0026lt;string,CommunityMember\u0026gt; members = 1; // The permissions of the chat CommunityPermissions permissions = 2; // The metadata of the chat ChatIdentity identity = 3; // The category of the chat string category_id = 4; // The position of chat in the display int32 position = 5; } message CommunityCategory { // The category id string category_id = 1; // The name of the category string name = 2; // The position of the category in the display int32 position = 3; } message CommunityInvitation { // Encrypted/unencrypted community description bytes community_description = 1; // The grant offered by the community bytes grant = 2; // The chat id requested to join string chat_id = 3; // The public key of the community bytes public_key = 4; } message CommunityRequestToJoin { // The Lamport timestamp of the request uint64 clock = 1; // The ENS name of the requester string ens_name = 2; // The chat id requested to join string chat_id = 3; // The public key of the community bytes community_id = 4; // The display name of the requester string display_name = 5; } message CommunityCancelRequestToJoin { // The Lamport timestamp of the request uint64 clock = 1; // The ENS name of the requester string ens_name = 2; // The chat id requested to join string chat_id = 3; // The public key of the community bytes community_id = 4; // The display name of the requester string display_name = 5; // Magnet uri for community history protocol string magnet_uri = 6; } message CommunityRequestToJoinResponse { // The Lamport timestamp of the request uint64 clock = 1; // The community description CommunityDescription community = 2; // If the request was accepted bool accepted = 3; // The grant offered by the community bytes grant = 4; // The community public key bytes community_id = 5; } message CommunityRequestToLeave { // The Lamport timestamp of the request uint64 clock = 1; // The community public key bytes community_id = 2; } message CommunityDescription { // The Lamport timestamp of the message uint64 clock = 1; // A mapping of members in the community to their roles map\u0026lt;string,CommunityMember\u0026gt; members = 2; // The permissions of the Community CommunityPermissions permissions = 3; // The metadata of the Community ChatIdentity identity = 5; // A mapping of chats to their details map\u0026lt;string,CommunityChat\u0026gt; chats = 6; // A list of banned members repeated string ban_list = 7; // A mapping of categories to their details map\u0026lt;string,CommunityCategory\u0026gt; categories = 8; // The admin settings of the Community CommunityAdminSettings admin_settings = 10; // If the community is encrypted bool encrypted = 13; // The list of tags repeated string tags = 14; } Note: The usage of the clock is described in the Clock section.
Content topic usage # \u0026ldquo;Content topic\u0026rdquo; refers to the field in 14/WAKU2-MESSAGE, further elaborated in 10/WAKU2.
Advertising a Community # The content topic that the community is advertised on MUST be derived from the public key of the community. The content topic MUST be the first four bytes of the keccak-256 hash of the compressed (33 bytes) public key of the community encoded into a hex string.
hash = hex(keccak256(encodeToHex(compressedPublicKey))) topicLen = 4 if len(hash) \u0026lt; topicLen { topicLen = len(hash) } var topic [4]byte for i = 0; i \u0026lt; topicLen; i++ { topic[i] = hash[i] } contentTopic = \u0026#34;/waku/1/0x\u0026#34; + topic + \u0026#34;/rfc26\u0026#34; Community channels/chats # The unique identifier for a community channel/chat is the chat id.
The content topic that Community channels/chats use MUST be the hex-encoded keccak-256 hash of the public key of the community concatenated with the chat id.
hash = hex(keccak256(encodeToHex(compressedPublicKey + chatId))) topicLen = 4 if len(hash) \u0026lt; topicLen { topicLen = len(hash) } var topic [4]byte for i = 0; i \u0026lt; topicLen; i++ { topic[i] = hash[i] } contentTopic = \u0026#34;/waku/1/0x\u0026#34; + topic + \u0026#34;/rfc26\u0026#34; Community event messages # Requests to leave, join, kick and ban, as well as key exchange messages, MUST be sent to the content topic derived from the public key of the community. The content topic MUST be the hex-encoded keccak-256 hash of the public key of the community.
hash = hex(keccak256(encodeToHex(publicKey))) topicLen = 4 if len(hash) \u0026lt; topicLen { topicLen = len(hash) } var topic [4]byte for i = 0; i \u0026lt; topicLen; i++ { topic[i] = hash[i] } contentTopic = \u0026#34;/waku/1/0x\u0026#34; + topic + \u0026#34;/rfc26\u0026#34; Community Management # The flows for Community management are as described below.
Community Creation Flow # The Community owner generates a public/private key pair. The Community owner configures the Community metadata, according to the wire format \u0026ldquo;CommunityDescription\u0026rdquo;. The Community owner publishes the Community metadata on a content topic derived from the public key of the Community. the Community metadata SHOULD be encrypted with the public key of the Community. The Community metadata MAY be sent during fixed intervals, to ensure that the Community metadata is available to members. The Community metadata SHOULD be sent every time the Community metadata is updated. The Community owner MAY advertise the Community out of band, by sharing the public key of the Community on other mediums of communication. Community Join Flow (peer requests to join a Community) # A peer and the Community owner establish a 1:1 chat as described in 55/STATUS-1TO1-CHAT. The peer requests to join a Community by sending a \u0026ldquo;CommunityRequestToJoin\u0026rdquo; message to the Community. At this point, the peer MAY send a \u0026ldquo;CommunityCancelRequestToJoin\u0026rdquo; message to cancel the request. The Community owner MAY accept or reject the request. If the request is accepted, the Community owner sends a \u0026ldquo;CommunityRequestToJoinResponse\u0026rdquo; message to the peer. The Community owner then adds the member to the Community metadata, and publishes the updated Community metadata. Community Join Flow (peer is invited to join a Community) # The Community owner and peer establish a 1:1 chat as described in 55/STATUS-1TO1-CHAT. The peer is invited to join a Community by the Community owner, by sending a \u0026ldquo;CommunityInvitation\u0026rdquo; message. The peer decrypts the \u0026ldquo;CommunityInvitation\u0026rdquo; message, and verifies the signature. The peer requests to join a Community by sending a \u0026ldquo;CommunityRequestToJoin\u0026rdquo; message to the Community. The Community owner MAY accept or reject the request. If the request is accepted, the Community owner sends a \u0026ldquo;CommunityRequestToJoinResponse\u0026rdquo; message to the peer. The Community owner then adds the member to the Community metadata, and publishes the updated Community metadata. Community Leave Flow # A member requests to leave a Community by sending a \u0026ldquo;CommunityRequestToLeave\u0026rdquo; message to the Community. The Community owner MAY accept or reject the request. If the request is accepted, the Community owner removes the member from the Community metadata, and publishes the updated Community metadata. Community Ban Flow # The Community owner adds a member to the ban list, revokes their grants, and publishes the updated Community metadata. If the Community is Private, Re-keying is performed between the members of the Community, to ensure that the banned member is unable to decrypt any messages. Waku Protocols # The following Waku protocols SHOULD be used to implement Status Communities -
11/WAKU2-RELAY - To send and receive messages 53/WAKU2-X3DH - To encrypt and decrypt messages 53/WAKU2-X3DH-SESSIONS - To handle session keys 14/WAKU2-MESSAGE - To wrap community messages in a Waku message 13/WAKU2-STORE - To store and retrieve messages for offline devices The following Waku protocols MAY be used to implement Status Communities -
12/WAKU2-FILTER - Content filtering for resource restricted devices 19/WAKU2-LIGHTPUSH - Allows Light clients to participate in the network Backups # The member MAY back up their local settings, by encrypting it with their public key, and sending it to a given content topic. The member MAY then rely on this backup to restore their local settings, in case of a data loss. This feature relies on 13/WAKU2-STORE for storing and retrieving messages.
Clock # The clock used in the wire format refers to the Lamport timestamp of the message. The Lamport timestamp is a logical clock that is used to determine the order of events in a distributed system. This allows ordering of messages in an asynchronous network where messages may be received out of order.
Security Considerations # The Community owner is a single point of failure. If the Community owner is compromised, the Community is compromised.
Follows the same security considerations as the 53/WAKU2-X3DH protocol.
Future work # To scale and optimize the Community management, the Community metadata should be stored on a decentralized storage system, and only the references to the Community metadata should be broadcasted. The following document describes this method in more detail - Optimizing the CommunityDescription dissemination
Token gating for communities
Sharding the content topic used for #Community Event Messages, since members of the community don\u0026rsquo;t need to receive all the control messages.
Copyright # Copyright and related rights waived via CC0.
References # 55/STATUS-1TO1-CHAT 53/WAKU2-X3DH 19/WAKU2-LIGHTPUSH 14/WAKU2-MESSAGE 10/WAKU2 11/WAKU2-RELAY 54/WAKU2-X3DH-SESSIONS 13/WAKU2-STORE 12/WAKU2-FILTER 19/WAKU2-LIGHTPUSH informative # community.go organisation-channels.md `}),e.add({id:47,href:"/spec/57/",title:"57/STATUS-Simple-Scaling",section:"Docs",content:` Abstract # This document describes how to scale 56/STATUS-COMMUNITIES as well as 55/STATUS-1TO1-CHAT using existing Waku v2 protocols and components. It also adds a few new aspects, where more sophisticated components are not yet researched and evaluated.
Note: (Parts of) this RFC will be deprecated in the future as we continue research to scale specific components in a way that aligns better with our principles of decentralization and protecting anonymity. This document informs about scaling at the current stage of research and shows it is practically possible. Practical feasibility is also a core goal for us. We believe in incremental improvement, i.e. having a working decentralized scaling solution with trade-offs is better than a fully centralized solution.
Background and Motivation # 56/STATUS-COMMUNITIES as well as 55/STATUS-1TO1-CHAT use Waku v2 protocols. Both use Waku content topics (see 23/WAKU2-TOPICS) for content based filtering.
Waku v2 currently has scaling limitations in two dimensions:
Messages that are part of a specific content topic have to be disseminated in a single mesh network (i.e. pubsub topic). This limits scaling the number of messages disseminated in a specific content topic, and by extension, the number of active nodes that are part of this content topic.
Scaling a large set of content topics requires distributing these over several mesh networks (which this document refers to as pubsub topic shards).
This document focuses on the second scaling dimension. With the scaling solutions discussed in this document, each content topics can have a large set of active users, but still has to fit in a single pubsub mesh.
Note: While it is possible to use the same content topic name on several shards, each node that is interested in this content topic has to be subscribed to all respective shards, which does not scale. Splitting content topics in a more sophisticated and efficient way will be part of a future document.
Relay Shards # Sharding the Waku Relay network is an integral part of scaling the Status app.
51/WAKU2-RELAY-SHARDING specifies shards clusters, which are sets of 1024 shards (separate pubsub mesh networks). Content topics specified by application protocols can be distributed over these shards. The Status app protocols are assigned to shard cluster 16, as defined in 52/WAKU2-RELAY-STATIC-SHARD-ALLOC.
51/WAKU2-RELAY-SHARDING specifies three sharding methods. This document uses static sharding, which leaves the distribution of content topics to application protocols, but takes care of shard discovery.
The 1024 shards within the main Status shard cluster are allocated as follows.
Shard Allocation # shard index usage 0 - 15 reserved 16 - 127 specific (large) communities 128 - 767 communities 768 - 895 1:1 chat 896 - 1023 media and control msgs Shard indices are mapped to pubsub topic names as follows (specified in 51/WAKU2-RELAY-SHARDING).
/waku/2/rs/\u0026lt;cluster_id\u0026gt;/\u0026lt;shard_number\u0026gt;
an example for the shard with index 18 in the Status shard cluster:
/waku/2/rs/16/18
In other words, the mesh network with the pubsub topic name /waku/2/rs/16/18 carries messages associated with shard 18 in the Status shard cluster.
Implementation Suggestion # The Waku implementation should offer an interface that allows Status nodes to subscribe to Status specific content topics like
subscribe(\u0026#34;/status/xyz\u0026#34;, 16, 18) The shard cluster index 16 can be kept in the Status app configuration, so that Status nodes can simply use
subscribe(\u0026#34;/status/xyz\u0026#34;, 18) which means: connect to the \u0026quot;status/xyz\u0026quot; content topic on shard 18 within the Status shard cluster.
Status Communities # In order to associate a community with a shard, the community description protobuf is extended by the field uint16 relay_shard_index = 15:
syntax = \u0026#34;proto3\u0026#34;; message CommunityDescription { // The Lamport timestamp of the message uint64 clock = 1; // A mapping of members in the community to their roles map\u0026lt;string,CommunityMember\u0026gt; members = 2; // The permissions of the Community CommunityPermissions permissions = 3; // The metadata of the Community ChatIdentity identity = 5; // A mapping of chats to their details map\u0026lt;string,CommunityChat\u0026gt; chats = 6; // A list of banned members repeated string ban_list = 7; // A mapping of categories to their details map\u0026lt;string,CommunityCategory\u0026gt; categories = 8; // The admin settings of the Community CommunityAdminSettings admin_settings = 10; // If the community is encrypted bool encrypted = 13; // The list of tags repeated string tags = 14; // index of the community\u0026#39;s shard within the Status shard cluster uint16 relay_shard_index = 15 } Note: Currently, Status app has allocated shared cluster 16 in 52/WAKU2-RELAY-STATIC-SHARD-ALLOC. Status app could allocate more shard clusters, for instance to establish a test net. We could add the shard cluster index to the community description as well. The recommendation for now is to keep it as a configuration option of the Status app.
Note: Once this RFC moves forward, the new community description protobuf fields should be mentioned in 56/STATUS-COMMUNITIES.
Status communities can be mapped to shards in two ways: static, and owner-based.
Static Mapping # With static mapping, communities are assigned a specific shard index within the Status shard cluster. This mapping is similar in nature to the shard cluster allocation in 52/WAKU2-RELAY-STATIC-SHARD-ALLOC. Shard indices allocated in that way are in the range 16 - 127. The Status CC community uses index 16 (not to confuse with shard cluster index 16, which is the Status shard cluster).
Owner Mapping # Note: This way of mapping will be specified post-MVP.
Community owners can choose to map their communities to any shard within the index range 128 - 767.
1:1 Chat # 55/STATUS-1TO1-CHAT uses partitioned topics to map 1:1 chats to a set of 5000 content topics. This document extends this mapping to 8192 content topics that are, in turn, mapped to 128 shards in the index range of 768 - 895.
contentPartitionsNum = 8192 contentPartition = mod(publicKey, contentPartitionsNum) partitionContentTopic = \u0026#34;contact-discovery-\u0026#34; + contentPartition partitionContentTopic = keccak256(partitionContentTopic) shardNum = 128 shardIndex = 768 + mod(publicKey, shardNum) Infrastructure Nodes # As described in 30/ADAPTIVE-NODES, Waku supports a continuum of node types with respect to available resources. Infrastructure nodes are powerful nodes that have a high bandwidth connection and a high up-time.
This document, which informs about simple ways of scaling Status over Waku, assumes the presence of a set of such infrastructure nodes in each shard. Infrastructure nodes are especially important for providing connectivity in the roll-out phase.
Infrastructure nodes are not limited to Status fleets, or nodes run by community owners. Anybody can run infrastructure nodes.
Statically-Mapped Communities # Infrastructure nodes are provided by the community owner, or by members of the respective community.
Owner-Mapped Communities # Infrastructure nodes are part of a subset of the shards in the range 128 - 767. Recommendations on choosing this subset will be added in a future version of this document.
Status fleet nodes make up a part of these infrastructure nodes.
1:1 chat # Infrastructure nodes are part of a subset of the shards in the range 768 - 985 (similar to owner-mapped communities). Recommendations on choosing this subset will be added in a future version of this document.
Desktop clients can choose to only use filter and lightpush.
Note: Discussion: I\u0026rsquo;d suggest to set this as the default for the MVP. The load on infrastructure nodes would not be higher, because they have to receive and relay each message anyways. This comes as a trade-off to anonymity and decentralization, but can significantly improve scaling. We still have k-anonymity because several chat pairs are mapped into one content topic. We could improve on this in the future, and research the applicability of PIR (private information retrieval) techniques in the future.
Infrastructure Shards # Waku messages are typically relayed in larger mesh networks comprised of nodes with varying resource profiles (see 30/ADAPTIVE-NODES). To maximise scaling, relaying of specific message types can be dedicated to shards where only infrastructure nodes with very strong resource profiles relay messages. This comes as a trade-off to decentralization.
Control Message Shards # To get the maximum scaling for select large communities for the Status scaling MVP, specific control messages that cause significant load (at a high user number) SHOULD be moved to a separate control message shard. These control messages comprise:
community description membership update backup community request to join response sync profile picture The relay functionality of control messages shards SHOULD be provided by infrastructure nodes. Desktop clients should use light protocols as the default for control message shards. Strong Desktop clients MAY opt in to support the relay network.
Each large community (in the index range of 16 - 127) can get its dedicated control message shard (in the index range 896 - 1023) if deemed necessary. The Status CC community uses shard 896 as its control message shard. This comes with trade-offs to decentralization and anonymity (see Security Considerations section).
Media Shards # Similar to control messages, media-heavy communities should use separate media shards (in the index range 896 - 1023) for disseminating messages with large media data. The Status CC community uses shard 897 as its media shard.
Infrastructure-focused Community # Large communities MAY choose to mainly rely on infrastructure nodes for all message transfers (not limited to control, and media messages). Desktop clients of such communities should use light protocols as the default. Strong Desktop clients MAY opt in to support the relay network.
Note: This is not planned for the MVP.
Light Protocols # Light protocols may be used to save bandwidth, at the (global) cost of not contributing to the network. Using light protocols is RECOMMENDED for resource restricted nodes, e.g. browsers, and devices that (temporarily) have a low bandwidth connection or a connection with usage-based billing.
Light protocols comprise
19/WAKU2-LIGHTPUSH for sending messages 12/WAKU2-FILTER for requesting messages with specific attributes 34/WAKU2-PEER-EXCHANGE for discovering peers Waku Archive # Archive nodes are Waku nodes that offer the Waku archive service via the Waku store protocol (13/WAKU2-STORE). They are part of a set of shards and store all messages disseminated in these shards. Nodes can request history messages via the 13/WAKU2-STORE.
The store service is not limited to a Status fleet. Anybody can run a Waku Archive node in the Status shards.
Note: There is no specification for discovering archive nodes associated with specific shards yet. Nodes expect archive nodes to store all messages, regardless of shard association.
The recommendation for the allocation of archive nodes to shards is similar to the allocation of infrastructure nodes to shards described above. In fact, the archive service can be offered by infrastructure nodes.
Discovery # Shard discovery is covered by 51/WAKU2-RELAY-SHARDING. This allows the Status app to abstract from the discovery process and simply address shards by their index.
Libp2p Rendezvous and Circuit-Relay # To make nodes behind restrictive NATs discoverable, this document suggests using libp2p rendezvous. Nodes can check whether they are behind a restrictive NAT using the libp2p AutoNAT protocol.
Note: The following will move into 51/WAKU2-RELAY-SHARDING, or 33/WAKU2-DISCV5: Nodes behind restrictive NATs SHOULD not announce their publicly unreachable address via 33/WAKU2-DISCV5 discovery.
It is RECOMMENDED that nodes that are part of the relay network also act as rendezvous points. This includes accepting register queries from peers, as well as answering rendezvous discover queries. Nodes MAY opt-out of the rendezvous functionality.
To allow nodes to initiate connections to peers behind restrictive NATs (after discovery via rendezvous), it is RECOMMENDED that nodes that are part of the Waku relay network also offer libp2p circuit relay functionality.
To minimize the load on circuit-relay nodes, nodes SHOULD
make use of the limiting functionality offered by the libp2p circuit relay protocols, and use DCUtR to upgrade to a direct connection. Nodes that do not announce themselves at all and only plan to use light protocols, MAY use rendezvous discovery instead of or along-side 34/WAKU2-PEER-EXCHANGE. For these nodes, rendezvous and 34/WAKU2-PEER-EXCHANGE offer the same functionality, but return node sets sampled in different ways. Using both can help increasing connectivity.
Nodes that are not behind restrictive NATs MAY register at rendezvous points, too; this helps increasing discoverability, and by extension connectivity. Such nodes SHOULD, however, not register at circuit relays.
Announcing Shard Participation # Registering a namespace via lib-p2p rendezvous is done via a register query:
REGISTER{my-app, {QmA, AddrA}} The app name, my-app is used to encode a single shard in the form:
\u0026lt;rs (utf8 encoded)\u0026gt; | \u0026lt;2-byte shard cluster index\u0026gt; | \u0026lt;2-byte shard index\u0026gt; Registering shard 2 in the Status shard cluster (with shard cluster index 16, see 52/WAKU2-RELAY-STATIC-SHARD-ALLOC), the register query would look like
REGISTER{0x727300100002, {QmA, AddrA}} Participation in further shards is registered with further queries; one register query per shard. (0x7273 is the encoding of rs.)
A discovery query for nodes that are part of this shard would look like
DISCOVER{ns: 0x727300100002} DoS Protection # Hereunder we describe the \u0026ldquo;opt-in message signing for DoS prevention\u0026rdquo; solution, designed ad hoc for Status MVP.
Since publishing messages to pubsub topics has no limits, anyone can publish messages at a very high rate and DoS the network. This would elevate the bandwidth consumption of all nodes subscribed to said pubsub topic, making it prohibitive (in terms of bandwidth) to be subscribed to it. In order to scale, we need some mechanism to prevent this from happening, otherwise all scaling efforts will be in vain. Since RLN is not ready yet, hereunder we describe a simpler approach designed ad hoc for Status use case, feasible to implement for the MVP and that validates some of the ideas that will evolve to solutions such as RLN.
With this approach, certain pubsub topics can be optionally configured to only accept messages signed with a given key, that only trusted entities know. This key can be pre-shared among a set of participants, that are trusted to make fair usage of the network, publishing messages at a reasonable rate/size. Note that this key can be shared/reused among multiple participants, and only one key is whitelisted per pubsub topic. This is an opt-in solution that operators can choose to deploy in their shards (i.e. pubsub topics), but it\u0026rsquo;s not enforced in the default one. Operators can freely choose how they want to generate, and distribute the public keys. It\u0026rsquo;s also their responsibility to handle the private key, sharing it with only trusted parties and keeping proper custody of it.
The following concepts are introduced:
private-key-topic: A private key of 32 bytes, that allows the holder to sign messages and it\u0026rsquo;s mapped to a protected-pubsub-topic. app-message-hash: Application WakuMessage hash, calculated as sha256(concat(pubsubTopic, payload, contentTopic)) with all elements in bytes. message-signature: ECDSA signature of application-message-hash using a given private-key-topic, 64 bytes. public-key-topic: The equivalent public key of private-key-topic. protected-pubsub-topic: Pubsub topic that only accepts messages that were signed with private-key-topic, where verify(message-signature, app-message-hash, public-key-topic) is only correct if the message-signature was produced by private-key-topic. See ECDSA signature verification algorithm. This solution introduces two roles:
Publisher: A node that knows the private-key-topic associated to public-key-topic, that can publish messages with a valid message-signature that are accepted and relayed by the nodes implementing this feature. Relayer: A node that knows the public-key-topic, which can be used to verify if the messages were signed with the equivalent private-key-topic. It allows distinguishing valid from invalid messages which protect the node against DoS attacks, assuming that the users of the key send messages of a reasonable size and rate. Note that a node can validate messages and relay them or not without knowing the private key. Design requirements (publisher) # A publisher that wants to send messages that are relayed in the network for a given protected-pubsub-topic shall:
be able to sign messages with the private-key-topic configured for that topic, producing a ECDSA signature of 64 bytes using deterministic signing complying with RFC 6979. include the signature of the app-message-hash (message-signature) that wishes to send in the WakuMessage meta field. The app-message-hash of the message shall be calculated as the sha256 hash of the following fields of the message:
sha256(concat(pubsubTopic, payload, contentTopic, timestamp, ephemeral)) Where fields are serialized into bytes using little-endian. Note that ephemeral is a boolean that is serialized to 0 if false and 1 if true.
Design requirements (relay) # Requirements for the relay are listed below:
A valid protected-pubsub-topic shall be configured with a public-key-topic, (derived from a private-key-topic). Note that the relay does not need to know the private key. For simplicity, there is just one key per topic. Since this approach has clear privacy implications, this configuration is not part of the waku protocol, but of the application. Requirements on the gossipsub validator:
Relay nodes should use the existing gossipsub validators that allow to Accept or Reject messages, according to the following criteria: If timestamp is not set (equals to 0) then Reject the message. If the timestamp is abs(current_timestamp-timestamp) \u0026gt; MessageWindowInSec then Reject the message. If meta is empty, Reject the message. If meta exists but its size is different than 64 bytes, Reject the message. If meta does not successfully verifies according to the ECDSA signature verification algorithm using public-key-topic and app-message-hash, then Reject the message. If and only if all above conditions are met then Accept the message. Other requirements:
The node shall keep metrics on the messages validation output, Accept or Reject. (Optional). To further strengthen DoS protection, gossipsub scoring can be used to trigger disconnections from peers sending multiple invalid messages. See P4 penalty. This protects each peer from DoS, since this score is used to trigger disconnections from nodes attempting to DoS them. Required changes # This solution is designed to be backward compatible so that nodes validating messages can coexist in the same topic with other nodes that don\u0026rsquo;t perform validation. But note that only nodes that perform message validation will be protected against DoS. Nodes wishing to opt-in this DoS protection feature shall:
Generate a private-key-topic and distribute it to a curated list of users, that are trusted to send messages at a reasonable rate. Redeploy the nodes, adding a new configuration where a protected-pubsub-topic is configured with a public-key-topic, used to verify the messages being relayed. Test vectors # Relay nodes complying with this specification shall accept the following message in the configured pubsub topic.
Given the following key pair:
private-key-topic = 5526a8990317c9b7b58d07843d270f9cd1d9aaee129294c1c478abf7261dd9e6 public-key-topic = 049c5fac802da41e07e6cdf51c3b9a6351ad5e65921527f2df5b7d59fd9b56ab02bab736cdcfc37f25095e78127500da371947217a8cd5186ab890ea866211c3f6 And the following message to send:
protected-pubsub-topic = pubsub-topic contentTopic = content-topic payload = 1A12E077D0E89F9CAC11FBBB6A676C86120B5AD3E248B1F180E98F15EE43D2DFCF62F00C92737B2FF6F59B3ABA02773314B991C41DC19ADB0AD8C17C8E26757B timestamp = 1683208172339052800 ephemeral = true The message hash and meta (aka signature) are calculated as follows.
app-message-hash = 662F8C20A335F170BD60ABC1F02AD66F0C6A6EE285DA2A53C95259E7937C0AE9 message.meta = 127FA211B2514F0E974A055392946DC1A14052182A6ABEFB8A6CD7C51DA1BF2E40595D28EF1A9488797C297EED3AAC45430005FB3A7F037BDD9FC4BD99F59E63 Using message.meta, the relay node shall calculate the app-message-hash of the received message using public-key-topic, and with the values above, the signature should be verified, making the node Accept the message and relaying it to other nodes in the network.
Owner-Mapped Communities # Basic idea: Tokenized load.
1:1 Chat # An idea we plan to explore in the future: Map 1:1 chats to community shards, if both A and B are part of the respective community. This increases k-anonymity and benefits from community DoS protection. It could be rate-limited with RLN.
Security/Privacy Considerations # This document makes several trade-offs to privacy and anonymity. Todo: elaborate. See 45/WAKU2-ADVERSARIAL-MODELS for information on Waku Anonymity.
Copyright # Copyright and related rights waived via CC0.
References # 56/STATUS-COMMUNITIES 55/STATUS-1TO1-CHAT 23/WAKU2-TOPICS 11/WAKU2-RELAY 51/WAKU2-RELAY-SHARDING 52/WAKU2-RELAY-STATIC-SHARD-ALLOC 30/ADAPTIVE-NODES 19/WAKU2-LIGHTPUSH 12/WAKU2-FILTER 34/WAKU2-PEER-EXCHANGE 13/WAKU2-STORE libp2p rendezvous libp2p AutoNAT protocol 33/WAKU2-DISCV5 libp2p circuit relay limiting DCUtR scoring 45/WAKU2-ADVERSARIAL-MODELS Informative # Circuit Relay 31/WAKU2-ENR `}),e.add({id:48,href:"/spec/58/",title:"58/RLN-V2",section:"Docs",content:` Abstract # The protocol specified in this document is an improvement of 32/RLN-V1, being more general construct, that allows to set various limits for an epoch (it\u0026rsquo;s 1 message per epoch in 32/RLN-V1) while remaining almost as simple as it predecessor. Moreover, it allows to set different rate-limits for different RLN app users based on some public data, e.g. stake or reputation.
Motivation # The main goal of this RFC is to generalize 32/RLN-V1 and expand its applications. There are two different subprotocols based on this protocol:
RLN-Same - RLN with the same rate-limit for all users; RLN-Diff - RLN that allows to set different rate-limits for different users. It is important to note that by using a large epoch limit value, users will be able to remain anonymous, because their internal_nullifiers will not be repeated until they exceed the limit.
Flow # As in 32/RLN-V1, the general flow can be described by three steps:
Registration Signaling Verification and slashing The two sub-protocols have different flows, and hence are defined separately.
Important note # All terms and parameters used remain the same as in 32/RLN-V1, more details here
RLN-Same flow # Registration # The registration process in the RLN-Same subprotocol does not differ from 32/RLN-V1.
Signalling # Proof generation # For proof generation, the user needs to submit the following fields to the circuit:
{ identity_secret: identity_secret_hash, path_elements: Merkle_proof.path_elements, identity_path_index: Merkle_proof.indices, x: signal_hash, message_id: message_id, external_nullifier: external_nullifier, message_limit: message_limit } Calculating output # The following fields are needed for proof output calculation:
{ identity_secret_hash: bigint, external_nullifier: bigint, message_id: bigint, x: bigint, } The output [y, internal_nullifier] is calculated in the following way:
a_0 = identity_secret_hash a_1 = poseidonHash([a0, external_nullifier, message_id]) y = a_0 + x * a_1 internal_nullifier = poseidonHash([a_1]) RLN-Diff flow # Registration # id_commitment in 32/RLN-V1 is equal to poseidonHash(identity_secret). The goal of RLN-Diff is to set different rate-limits for different users. It follows that id_commitment must somehow depend on the user_message_limit parameter, where 0 \u0026lt;= user_message_limit \u0026lt;= message_limit. There are few ways to do that:
Sending identity_secret_hash = poseidonHash(identity_secret, userMessageLimit) and zk proof that user_message_limit is valid (is in the right range). This approach requires zkSNARK verification, which is an expensive operation on the blockchain. Sending the same identity_secret_hash as in 32/RLN-V1 (poseidonHash(identity_secret)) and a user_message_limit publicly to a server or smart-contract where rate_commitment = poseidonHash(identity_secret_hash, userMessageLimit) is calculated. The leaves in the membership Merkle tree would be the rate_commitments of the users. This approach requires additional hashing in the Circuit, but it eliminates the need for zk proof verification for the registration. Both methods are correct, and the choice of the method is left to the implementer. It is recommended to use second method for the reasons already described. The following flow description will also be based on the second method.
Signalling # Proof generation # For proof generation, the user need to submit the following fields to the circuit:
{ identity_secret: identity_secret_hash, path_elements: Merkle_proof.path_elements, identity_path_index: Merkle_proof.indices, x: signal_hash, message_id: message_id, external_nullifier: external_nullifier, user_message_limit: message_limit } Calculating output # The Output is calculated in the same way as the RLN-Same sub-protocol.
Verification and slashing # Verification and slashing in both subprotocols remain the same as in 32/RLN-V1. The only difference that may arise is the message_limit check in RLN-Same, since it is now a public input of the Circuit.
ZK Circuits specification # The design of the 32/RLN-V1 circuits is different from the circuits of this protocol. RLN-v2 requires additional algebraic constraints. The membership proof and Shamir\u0026rsquo;s Secret Sharing constraints remain unchanged.
The ZK Circuit is implemented using a Groth-16 ZK-SNARK, using the circomlib library. Both schemes contain compile-time constants/system parameters:
DEPTH - depth of membership Merkle tree LIMIT_BIT_SIZE - bit size of limit numbers, e.g. for the 16 - maximum limit number is 65535. The main difference of the protocol is that instead of a new polynomial (a new value a_1) for a new epoch, a new polynomial is generated for each message. The user assigns an identifier to each message; the main requirement is that this identifier be in the range from 1 to limit. This is proven using range constraints.
RLN-Same circuit # Circuit parameters # Public Inputs
x external_nullifier message_limit - limit per epoch Private Inputs
identity_secret_hash path_elements identity_path_index message_id Outputs
y root internal_nullifier RLN-Diff circuit # In the RLN-Diff scheme, instead of the public parameter message_limit, a parameter is used that is set for each user during registration (user_message_limit); the message_id value is compared to it in the same way as it is compared to message_limit in the case of RLN-Same.
Circuit parameters # Public Inputs
x external_nullifier Private Inputs
identity_secret_hash path_elements identity_path_index message_id user_message_limit Outputs
y root internal_nullifier Appendix A: Security considerations # Although there are changes in the circuits, this spec inherits all the security considerations of 32/RLN-V1.
Copyright # Copyright and related rights waived via CC0.
References # [1] https://zkresear.ch/t/rate-limit-nullifier-v2-circuits/102 [2] https://github.com/Rate-Limiting-Nullifier/rln-circuits-v2 [3] https://rfc.vac.dev/spec/32/#technical-overview `}),e.add({id:49,href:"/spec/6/",title:"6/WAKU1",section:"Docs",content:`This specification describes the format of Waku packets within the ÐΞVp2p Wire Protocol. This spec substitutes EIP-627. Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic envelopes (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
Motivation # Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku packets in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
Definitions # Term Definition Batch Ack An abbreviated term for Batch Acknowledgment Light node A Waku node that does not forward any envelopes through the Messages packet. Envelope Messages sent and received by Waku nodes. Described in ABNF spec waku-envelope Node Some process that is able to communicate for Waku. Underlying Transports and Prerequisites # Use of DevP2P # For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
This protocol needs to advertise the waku/1 capability.
Gossip based routing # In Whisper, envelopes are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Envelopes are eligible for retransmission until their TTL expires. A node SHOULD relay envelopes to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a mailserver response. A node SHOULD NOT send an envelope to a peer that it has already sent before.
Maximum Packet Size # Nodes SHOULD limit the maximum size of both packets and envelopes. If a packet or envelope exceeds its limit, it MUST be dropped.
RLPx Packet Size - This size MUST be checked before a message is decoded. Waku Envelope Size - Each envelope contained in an RLPx packet MUST then separately be checked against the maximum envelope size. Clients MAY use their own maximum packet and envelope sizes. The default values are 1.5mb for the RLPx Packet and 1mb for a Waku envelope.
Wire Specification # Use of RLPx transport protocol # All Waku packets are sent as devp2p RLPx transport protocol, version 51 packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See informal RLP spec and the Ethereum Yellow Paper, appendix B for more details on RLP.
Waku is a RLPx subprotocol called waku with version 0. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of waku, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
ABNF specification # Using Augmented Backus-Naur form (ABNF) we have the following format:
; Packet codes 0 - 127 are reserved for Waku protocol packet-code = 1*3DIGIT ; rate limits per packet packet-limit-ip = 1*DIGIT packet-limit-peerid = 1*DIGIT packet-limit-topic = 1*DIGIT ; rate limits by size in bytes bytes-limit-ip = 1*DIGIT bytes-limit-peerid = 1*DIGIT bytes-limit-topic = 1*DIGIT packet-rate-limits = \u0026#34;[\u0026#34; packet-limit-ip packet-limit-peerid packet-limit-topic \u0026#34;]\u0026#34; bytes-rate-limits = \u0026#34;[\u0026#34; bytes-limit-ip bytes-limit-peerid bytes-limit-topic \u0026#34;]\u0026#34; pow-requirement-key = 0 bloom-filter-key = 1 light-node-key = 2 confirmations-enabled-key = 3 packet-rate-limits-key = 4 topic-interest-key = 5 bytes-rate-limits-key = 6 status-options = \u0026#34;[\u0026#34; [ pow-requirement-key pow-requirement ] [ bloom-filter-key bloom-filter ] [ light-node-key light-node ] [ confirmations-enabled-key confirmations-enabled ] [ packet-rate-limits-key packet-rate-limits ] [ topic-interest-key topic-interest ] [ bytes-limits-key bytes-rate-limits ] \u0026#34;]\u0026#34; status = status-options status-update = status-options confirmations-enabled = BIT light-node = BIT ; pow is \u0026#34;a single floating point value of PoW. ; This value is the IEEE 754 binary representation ; of a 64-bit floating point number packed as a uint64. ; Values of qNAN, sNAN, INF and -INF are not allowed. ; Negative values are also not allowed.\u0026#34; pow = 1*DIGIT \u0026#34;.\u0026#34; 1*DIGIT pow-requirement = pow ; bloom filter is \u0026#34;a byte array\u0026#34; bloom-filter = *OCTET waku-envelope = \u0026#34;[\u0026#34; expiry ttl topic data nonce \u0026#34;]\u0026#34; ; List of topics interested in topic-interest = \u0026#34;[\u0026#34; *10000topic \u0026#34;]\u0026#34; ; 4 bytes (UNIX time in seconds) expiry = 4OCTET ; 4 bytes (time-to-live in seconds) ttl = 4OCTET ; 4 bytes of arbitrary data topic = 4OCTET ; byte array of arbitrary size ; (contains encrypted payload) data = *OCTET ; 8 bytes of arbitrary data ; (used for PoW calculation) nonce = 8OCTET messages = 1*waku-envelope ; version of the confirmation packet version = 1*DIGIT ; keccak256 hash of the envelopes batch data (raw bytes) ; for which the confirmation is sent hash = *OCTET hasherror = *OCTET ; error code code = 1*DIGIT ; a descriptive error message description = *ALPHA error = \u0026#34;[\u0026#34; hasherror code description \u0026#34;]\u0026#34; errors = *error response = \u0026#34;[\u0026#34; hash errors \u0026#34;]\u0026#34; confirmation = \u0026#34;[\u0026#34; version response \u0026#34;]\u0026#34; ; message confirmation packet types batch-ack = confirmation message-response = confirmation ; mail server / client specific p2p-request = waku-envelope p2p-message = 1*waku-envelope p2p-request-complete = *OCTET ; packet-format needs to be paired with its ; corresponding packet-format packet-format = \u0026#34;[\u0026#34; packet-code packet-format \u0026#34;]\u0026#34; required-packet = 0 status / 1 messages / 22 status-update / optional-packet = 11 batch-ack / 12 message-response / 126 p2p-request-complete / 126 p2p-request / 127 p2p-message packet = \u0026#34;[\u0026#34; required-packet [ optional-packet ] \u0026#34;]\u0026#34; All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from 0x00.
Packet Codes # The packet codes reserved for Waku protocol: 0 - 127.
Packets with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
The Waku sub-protocol MUST support the following packet codes:
Name Int Value Status 0 Messages 1 Status Update 22 The following message codes are optional, but they are reserved for specific purpose.
Name Int Value Comment Batch Ack 11 Message Response 12 P2P Request Complete 125 P2P Request 126 P2P Message 127 Packet usage # Status # The Status packet serves as a Waku handshake and peers MUST exchange this packet upon connection. It MUST be sent after the RLPx handshake and prior to any other Waku packets.
A Waku node MUST await the Status packet from a peer before engaging in other Waku protocol activity with that peer. When a node does not receive the Status packet from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
Upon retrieval of the Status packet, the node SHOULD validate the packet received and validated the Status packet. Note that its peer might not be in the same state.
When a node is receiving other Waku packets from a peer before a Status packet is received, the node MUST ignore these packets and SHOULD disconnect from that peer. Status packets received after the handshake is completed MUST also be ignored.
The Status packet MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
Messages # This packet is used for sending the standard Waku envelopes.
Status Update # The Status Update packet is used to communicate an update of the settings of the node. The format is the same as the Status packet, all fields are optional. If none of the options are specified the packet MUST be ignored and considered a noop. Fields that are omitted are considered unchanged, fields that haven\u0026rsquo;t changed SHOULD not be transmitted.
PoW Requirement Field # When PoW Requirement is updated, peers MUST NOT deliver envelopes with PoW lower than the PoW Requirement specified.
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by envelope size and TTL:
PoW = (2**BestBit) / (size * TTL) PoW calculation:
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field. fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce) fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl) where size is the size of the RLP-encoded envelope, excluding env_nonce field (size of short_rlp(envelope)).
Bloom Filter Field # The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
LET D[*] = 0 FOREACH i IN { 0, 1, 2 } DO LET n = S[i] IF S[3] \u0026amp; (2 ** i) THEN n += 256 D[n] = 1 END FOR A full bloom filter (all the bits set to 1) means that the node is to be considered a Full Node and it will accept any topic.
If both topic interest and bloom filter are specified, topic interest always takes precedence and bloom filter MUST be ignored.
If only bloom filter is specified, the current topic interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
Topic Interest Field # Topic interest is used to share a node\u0026rsquo;s interest in envelopes with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
If only topic interest is specified, the current bloom filter MUST be discarded and only the updated topic interest MUST be used when forwarding or posting envelopes.
An empty array signals that the node is not currently interested in receiving any envelope.
Rate Limits Field # Rate limits is used to inform other nodes of their self defined rate limits.
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
If a peer exceeds node\u0026rsquo;s rate limits, the connection between them MAY be dropped.
Each node SHOULD broadcast its rate limits to its peers using the status-update packet. The rate limits MAY also be sent as an optional parameter in the handshake.
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer\u0026rsquo;s rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
Two rate limits strategies are applied:
Number of packets per second Size of packets (in bytes) per second Both strategies SHOULD be applied per IP address, peer id and topic.
The size limit SHOULD be greater or equal than the maximum packet size.
Light Node Field # When the node\u0026rsquo;s light-node field is set to true, the node SHOULD NOT forward Envelopes from its peers.
A node connected to a peer with the light-node field set to true MUST NOT depend on the peer for forwarding Envelopes.
Confirmations Enabled Field # When the node\u0026rsquo;s confirmations-enabled field is set to true, the node SHOULD send message confirmations to its peers.
Batch Ack and Message Response # Message confirmations tell a node that an envelope originating from it has been received by its peers, allowing a node to know whether an envelope has or has not been received.
A node MAY send a message confirmation for any batch of envelopes received with a Messages packet (0x01).
A message confirmation is sent using Batch Ack packet (0x0B) or Message Response packet (0x0C). The message confirmation is specified in the ABNF specification.
The current version in the confirmation is 1.
The supported error codes:
1: time sync error which happens when an envelope is too old or was created in the future (typically because of an unsynchronized clock of a node). The drawback of sending message confirmations is that it increases the noise in the network because for each sent envelope, a corresponding confirmation is broadcast by one or more peers.
P2P Request # This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting historic (expired) envelopes from the Waku Mail Server.
P2P Message # This packet is used for sending the peer-to-peer envelopes, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of historic (expired) envelopes, which is otherwise not allowed.
P2P Request Complete # This packet is used to indicate that all envelopes, requested earlier with a P2P Request packet (0x7E), have been sent via one or more P2P Message packets (0x7F).
The content of the packet is explained in the Waku Mail Server specification.
Payload Encryption # Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
Packet code Rationale # Packet codes 0x00 and 0x01 are already used in all Waku / Whisper versions. Packet code 0x02 and 0x03 were previously used in Whisper but are deprecated as of Waku v0.4
Packet code 0x22 is used to dynamically change the settings of a node.
Packet codes 0x7E and 0x7F may be used to implement Waku Mail Server and Client. Without the P2P Message packet it would be impossible to deliver the historic envelopes, since they will be recognized as expired, and the peer will be disconnected for violating the Waku protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
Additional capabilities # Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
Additionally there is the capability of a mailserver which is documented in its on specification.
Light node # The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
Light nodes MUST NOT forward any incoming envelopes, they MUST only send their own envelopes. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in envelopes being dropped between the two.
Light nodes are identified by the light_node value in the Status packet.
Accounting for resources (experimental) # Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm\u0026rsquo;s SWAP protocol, and works by doing pairwise accounting for resources.
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
peer sent received peer1 0 123 peer2 10 40 In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
Upgradability and Compatibility # General principles and policy # The currently advertised capability is waku/1. This needs to be advertised in the hello ÐΞVp2p packet. If a node supports multiple versions of waku, those needs to be explicitly advertised. For example if both waku/0 and waku/1 are supported, both waku/0 and waku/1 MUST be advertised.
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
Waku aims to be compatible with previous and future versions.
In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
To achieve this, we employ the following two general strategies:
a) Accretion (including protocol negotiation) over changing data b) When we want to change things, we give it a new name (for example, a version number). Examples:
We enable bridging between shh/6 and waku/1 until such a time as when we are ready to gracefully drop support for shh/6 (1, 2, 3). When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3). To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3) When we we want to provide a new set of packets that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3) Backwards Compatibility # Waku is a different subprotocol from Whisper so it isn\u0026rsquo;t directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
Waku-Whisper bridging # waku/1 and shh/6 are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
Roles:
Waku client A, only Waku capability Whisper client B, only Whisper capability WakuWhisper bridge C, both Waku and Whisper capability Flow:
A posts envelope; B posts envelope. C picks up envelope from A and B and relays them both to Waku and Whisper. A receives envelope on Waku; B on Whisper. Note: This flow means if another bridge C1 is active, we might get duplicate relaying for a envelope between C1 and C2. I.e. Whisper(\u0026lt;\u0026gt;Waku\u0026lt;\u0026gt;Whisper)\u0026lt;\u0026gt;Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
Forward Compatibility # It is desirable to have a strategy for maintaining forward compatibility between waku/1 and future version of waku. Here we outline some concerns and strategy for this.
Connecting to nodes with multiple versions: The way this SHOULD be accomplished is by negotiating the versions of subprotocols, within the hello packet nodes transmit their capabilities along with a version. The highest common version should then be used. Adding new packet codes: New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet. Adding new options in status-options: New options can be added to the status-options association list in the status and status-update packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving status-options with unknown option keys. Appendix A: Security considerations # There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as mailservers can be found in their respective specifications.
Scalability and UX # Bandwidth usage: # In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described here.
Gossip-based routing: # Use of gossip-based routing doesn\u0026rsquo;t necessarily scale. It means each node can see an envelope multiple times, and having too many light nodes can cause propagation probability that is too low. See Whisper vs PSS for more and a possible Kademlia based alternative.
Lack of incentives: # Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
Privacy # Light node privacy: # The main privacy concern with a light node is that it has to reveal its topic interests (in addition to its IP/ID) to its directed peers. This is because when a light node publishes an envelope, its directed peers will know that the light node owns that envelope (as light nodes do not relay other envelopes). Therefore, the directed peers of a light node can make assumptions about what envelopes (topics) the light node is interested in.
Mailserver client privacy: # A mailserver client fetches archival envelopes from a mailserver through a direct connection. In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver. The collection of such information allows the mailserver to link clients\u0026rsquo; IP/IDs to their topic interests and build a profile for each client over time. As such, the mailserver client has to trust the mailserver with this level of information.
Bloom filter privacy: # By having a bloom filter where only the topics you are interested in are set, you reveal which envelopes you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the Anonymity trilemma.
Privacy guarantees not rigorous: # Privacy for Whisper / Waku haven\u0026rsquo;t been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
Topic hygiene: # Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
Spam resistance # PoW bad for heterogeneous devices:
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
Censorship resistance # Devp2p TCP port blockable:
By default Devp2p runs on port 30303, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port 80 or 443, but there are still outstanding issues. See libp2p and Tor\u0026rsquo;s Pluggable Transport for how this can be improved.
Appendix B: Implementation Notes # Implementation Matrix # Client Spec supported Details Status-go 0.5 details Nim-waku 1.0 details Recommendations for clients # Notes useful for implementing Waku mode.
Avoid duplicate envelopes
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
Topic specific recommendations
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
Node discovery # Resource restricted devices SHOULD use EIP-1459 to discover nodes.
Known static nodes MAY also be used.
Changelog # Initial Release # Add section on P2P Request Complete packet and update packet code table. Correct the header hierarchy for the status-options fields. Consistent use of the words packet, message and envelope. Added section on max packet size Complete the ABNF specification and minor ABNF fixes. Version 1.1 # Released June 09, 2020
Add rate limit per bytes Version 1.0 # Released April 21,2020
Removed version from handshake Changed RLP keys from 48,49.. to 0,1.. Upgraded to waku/1 Version 0.6 # Released April 21,2020
Mark spec as Deprecated mode in terms of its lifecycle. Version 0.5 # Released March 17,2020
Clarify the preferred way of handling unknown keys in the status-options association list. Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior Version 0.4 # Released February 21, 2020.
Simplify implementation matrix with latest state Introduces a new required packet code Status Code (0x22) for communicating option changes Deprecates the following packet codes: PoW Requirement (0x02), Bloom Filter (0x03), Rate limits (0x20), Topic interest (0x21) - all superseded by the new Status Code (0x22) Increased topic-interest capacity from 1000 to 10000 Version 0.3 # Released February 13, 2020.
Recommend DNS based node discovery over other Discovery methods. Mark spec as Draft mode in terms of its lifecycle. Simplify Changelog and misc formatting. Handshake/Status packet not compatible with shh/6 nodes; specifying options as association list. Include topic-interest in Status handshake. Upgradability policy. topic-interest packet code. Version 0.2 # Released December 10, 2019.
General style improvements. Fix ABNF grammar. Mailserver requesting/receiving. New packet codes: topic-interest (experimental), rate limits (experimental). More details on handshake modifications. Accounting for resources mode (experimental) Appendix with security considerations: scalability and UX, privacy, and spam resistance. Appendix with implementation notes and implementation matrix across various clients with breakdown per capability. More details on handshake and parameters. Describe rate limits in more detail. More details on mailserver and mail client API. Accounting for resources mode (very experimental). Clarify differences with Whisper. Version 0.1 # Initial version. Released November 21, 2019.
Differences between shh/6 and waku/1 # Summary of main differences between this spec and Whisper v6, as described in EIP-627:
RLPx subprotocol is changed from shh/6 to waku/1. Light node capability is added. Optional rate limiting is added. Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits Mail Server and Mail Client functionality is now part of the specification. P2P Message packet contains a list of envelopes instead of a single envelope. Copyright # Copyright and related rights waived via CC0.
Footnotes # Felix Lange et al. The RLPx Transport Protocol. Ethereum.\u0026#160;\u0026#x21a9;\u0026#xfe0e;
`}),e.add({id:50,href:"/spec/61/",title:"61/STATUS-Community-History-Archives",section:"Docs",content:` Abstract # Messages are stored permanently by store nodes (13/WAKU2-STORE) for up to a certain configurable period of time, limited by the overall storage provided by a store node. Messages older than that period are no longer provided by store nodes, making it impossible for other nodes to request historical messages that go beyond that time range. This raises issues in the case of Status communities, where recently joined members of a community are not able to request complete message histories of the community channels.
This specification describes how Control Nodes (which are specific nodes in Status communities) archive historical message data of their communities, beyond the time range limit provided by Store Nodes using the BitTorrent protocol. It also describes how the archives are distributed to community members via the Status network, so they can fetch them and gain access to a complete message history.
Terminology # The following terminology is used throughout this specification. Notice that some actors listed here are nodes that operate in Waku networks only, while others operate in the Status communities layer):
Name References Waku node An Waku node (10/WAKU2) that implements 11/WAKU2-RELAY Store node A Waku node that implements 13/WAKU2-STORE Waku network A group of Waku nodes forming a graph, connected via 11/WAKU2-RELAY Status user An Status account that is used in a Status consumer product, such as Status Mobile or Status Desktop Status node A Status client run by a Status application Control node A Status node that owns the private key for a Status community Community member A Status user that is part of a Status community, not owning the private key of the community Community member node A Status node with message archive capabilities enabled, run by a community member Live messages Waku messages received through the Waku network BitTorrent client A program implementing the BitTorrent protocol Torrent/Torrent file A file containing metadata about data to be downloaded by BitTorrent clients Magnet link A link encoding the metadata provided by a torrent file (Magnet URI scheme) Requirements / Assumptions # This specification has the following assumptions:
Store nodes(13/WAKU2-STORE) are available 24/7, ensuring constant live message availability. The storage time range limit is 30 days. Store nodes have enough storage to persist historical messages for up to 30 days. No store nodes have storage to persist historical messages older than 30 days. All nodes are honest. The network is reliable. Furthermore, it assumes that:
Control nodes have enough storage to persist historical messages older than 30 days. Control nodes provide archives with historical messages at least every 30 days. Control nodes receive all community messages. Control nodes are honest. Control nodes know at least one store node from which it can query historical messages. These assumptions are less than ideal and will be enhanced in future work. This forum discussion provides more details.
Overview # The following is a high-level overview of the user flow and features this specification describes. For more detailed descriptions, read the dedicated sections in this specification.
Serving community history archives # Control nodes go through the following (high level) process to provide community members with message histories:
Community owner creates a Status community (previously known as org channels) which makes its node a Control node. Community owner enables message archive capabilities (on by default but can be turned off as well - see UI feature spec). A special type of channel to exchange metadata about the archival data is created, this channel is not visible in the user interface. Community owner invites community members. Control node receives messages published in channels and stores them into a local database. After 7 days, the control node exports and compresses last 7 days worth of messages from database and bundles it together with a message archive index into a torrent, from which it then creates a magnet link (Magnet URI scheme, Extensions for Peers to Send Metadata Files). Control node sends the magnet link created in step 6 to community members via special channel created in step 3 through the Waku network. Every subsequent 7 days, steps 6 and 7 are repeated and the new message archive data is appended to the previously created message archive data. Serving archives for missed messages # If the control node goes offline (where \u0026ldquo;offline\u0026rdquo; means, the control node\u0026rsquo;s main process is no longer running), it MUST go through the following process:
Control node restarts Control node requests messages from store nodes for the missed time range for all channels in their community All missed messages are stored into control node\u0026rsquo;s local message database If 7 or more days have elapsed since the last message history torrent was created, the control node will perform step 6 and 7 of Serving community history archives for every 7 days worth of messages in the missed time range (e.g. if the node was offline for 30 days, it will create 4 message history archives) Receiving community history archives # Community member nodes go through the following (high level) process to fetch and restore community message histories:
User joins community and becomes community member (see org channels spec) By joining a community, member nodes automatically subscribe to special channel for message archive metadata exchange provided by the community Member node requests live message history (last 30 days) of all the community channels including the special channel from store nodes Member node receives Waku message (14/WAKU2-MESSAGE) that contains the metadata magnet link from the special channel Member node extracts the magnet link from the Waku message and passes it to torrent client Member node downloads message archive index file and determines which message archives are not downloaded yet (all or some) Member node fetches missing message archive data via torrent Member node unpacks and decompresses message archive data to then hydrate its local database, deleting any messages for that community that the database previously stored in the same time range as covered by the message history archive Storing live messages # For archival data serving, the control node MUST store live messages as 14/WAKU2-MESSAGE. This is in addition to their database of application messages. This is required to provide confidentiality, authenticity, and integrity of message data distributed via the BitTorrent layer, and later validated by community members when they unpack message history archives.
Control nodes SHOULD remove those messages from their local databases once they are older than 30 days and after they have been turned into message archives and distributed to the BitTorrent network.
Exporting messages for bundling # Control nodes export Waku messages from their local database for creating and bundling history archives using the following criteria:
Waku messages to be exported MUST have a contentTopic that match any of the topics of the community channels Waku messages to be exported MUST have a timestamp that lies within a time range of 7 days The timestamp is determined by the context in which the control node attempts to create a message history archives as described below:
The control node attempts to create an archive periodically for the past seven days (including the current day). In this case, the timestamp has to lie within those 7 days. The control node has been offline (control node\u0026rsquo;s main process has stopped and needs restart) and attempts to create archives for all the live messages it has missed since it went offline. In this case, the timestamp has to lie within the day the latest message was received and the current day. Exported messages MUST be restored as 14/WAKU2-MESSAGE for bundling. Waku messages that are older than 30 days and have been exported for bundling can be removed from the control node\u0026rsquo;s database (control nodes still maintain a database of application messages).
Message history archives # Message history archives are represented as WakuMessageArchive and created from Waku messages exported from the local database. Message history archives are implemented using the following protocol buffer.
WakuMessageHistoryArchive # The from field SHOULD contain a timestamp of the time range\u0026rsquo;s lower bound. The type parallels the timestamp of WakuMessage.
The to field SHOULD contain a timestamp of the time range\u0026rsquo;s the higher bound.
The contentTopic field MUST contain a list of all communiity channel topics.
The messages field MUST contain all messages that belong into the archive given its from, to and contentTopic fields.
The padding field MUST contain the amount of zero bytes needed so that the overall byte size of the protobuf encoded WakuMessageArchive is a multiple of the pieceLength used to divide the message archive data into pieces. This is needed for seamless encoding and decoding of archival data in interation with BitTorrent as explained in creating message archive torrents.
syntax = \u0026#34;proto3\u0026#34; message WakuMessageArchiveMetadata { uint8 version = 1 uint64 from = 2 uint64 to = 3 repeated string contentTopic = 4 } message WakuMessageArchive { uint8 version = 1 WakuMessageArchiveMetadata metadata = 2 repeated WakuMessage messages = 3 // \`WakuMessage\` is provided by 14/WAKU2-MESSAGE bytes padding = 4 } Message history archive index # Control nodes MUST provide message archives for the entire community history. The entirey history consists of a set of WakuMessageArchive\u0026rsquo;s where each archive contains a subset of historical WakuMessages for a time range of seven days. All the WakuMessageArchives are concatenated into a single file as a byte string (see Ensuring reproducible data pieces).
Control nodes MUST create a message history archive index (WakuMessageArchiveIndex) with metadata that allows receiving nodes to only fetch the message history archives they are interested in.
WakuMessageArchiveIndex # A WakuMessageArchiveIndex is a map where the key is the KECCAK-256 hash of the WakuMessageArchiveIndexMetadata derived from a 7-day archive and the value is an instance of that WakuMessageArchiveIndexMetadata corresponding to that archive.
The offset field MUST contain the position at which the message history archive starts in the byte string of the total message archive data. This MUST be the sum of the length of all previously created message archives in bytes (see Creating message archive torrents).
syntax = \u0026#34;proto3\u0026#34; message WakuMessageArchiveIndexMetadata { uint8 version = 1 WakuMessageArchiveMetadata metadata = 2 uint64 offset = 3 uint64 num_pieces = 4 } message WakuMessageArchiveIndex { map\u0026lt;string, WakuMessageArchiveIndexMetadata\u0026gt; archives = 1 } The control node MUST update the WakuMessageArchiveIndex every time it creates one or more WakuMessageArchives and bundle it into a new torrent. For every created WakuMessageArchive, there MUST be a WakuMessageArchiveIndexMetadata entry in the archives field WakuMessageArchiveIndex.
Creating message archive torrents # Control nodes MUST create a torrent file (\u0026ldquo;torrent\u0026rdquo;) containing metadata to all message history archives. To create a torrent file, and later serve the message archive data in the BitTorrent network, control nodes MUST store the necessary data in dedicated files on the file system.
A torrent\u0026rsquo;s source folder MUST contain the following two files:
data - Contains all protobuf encoded WakuMessageArchive\u0026rsquo;s (as bit strings) concatenated in ascending order based on their time index - Contains the protobuf encoded WakuMessageArchiveIndex Control nodes SHOULD store these files in a dedicated folder that is identifiable via the community id.
Ensuring reproducible data pieces # The control node MUST ensure that the byte string resulting from the protobuf encoded data is equal to the byte string data from the previously generated message archive torrent, plus the data of the latest 7 days worth of messages encoded as WakuMessageArchive. Therefore, the size of data grows every seven days as it\u0026rsquo;s append only.
The control nodes also MUST ensure that the byte size of every individual WakuMessageArchive encoded protobuf is a multiple of pieceLength: ??? (TODO) using the padding field. If the protobuf encoded \u0026lsquo;WakuMessageArchiveis not a multiple ofpieceLength, its paddingfield MUST be filled with zero bytes and theWakuMessageArchiveMUST be re-encoded until its size becomes multiple ofpieceLength\`.
This is necessary because the content of the data file will be split into pieces of pieceLength when the torrent file is created, and the SHA1 hash of every piece is then stored in the torrent file and later used by other nodes to request the data for each individual data piece.
By fitting message archives into a multiple of pieceLength and ensuring they fill possible remaining space with zero bytes, control nodes prevent the next message archive to occupy that remaining space of the last piece, which will result in a different SHA1 hash for that piece.
Example: Without padding # Let WakuMessageArchive \u0026ldquo;A1\u0026rdquo; be of size 20 bytes:
0 11 22 33 44 55 66 77 88 99 10 11 12 13 14 15 16 17 18 19 With a pieceLength of 10 bytes, A1 will fit into 20 / 10 = 2 pieces:
0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123 10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456 Example: With padding # Let WakuMessageArchive \u0026ldquo;A2\u0026rdquo; be of size 21 bytes:
0 11 22 33 44 55 66 77 88 99 10 11 12 13 14 15 16 17 18 19 20 With a pieceLength of 10 bytes, A2 will fit into 21 / 10 = 2 pieces. The remainder will introduce a third piece:
0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123 10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456 20 // piece[2] SHA1: 0x789 The next WakuMessageArchive \u0026ldquo;A3\u0026rdquo; will be appended (\u0026quot;#3\u0026quot;) to the existing data and occupy the remaining space of the third data piece. The piece at index 2 will now produce a different SHA1 hash:
0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123 10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456 20 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[2] SHA1: 0xeef #3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[3] By filling up the remaining space of the third piece with A2 using its padding field, it is guaranteed that its SHA1 will stay the same:
0 11 22 33 44 55 66 77 88 99 // piece[0] SHA1: 0x123 10 11 12 13 14 15 16 17 18 19 // piece[1] SHA1: 0x456 20 0 0 0 0 0 0 0 0 0 // piece[2] SHA1: 0x999 #3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[3] #3 #3 #3 #3 #3 #3 #3 #3 #3 #3 // piece[4] Seeding message history archives # The control node MUST seed the generated torrent until a new WakuMessageArchive is created.
The control node SHOULD NOT seed torrents for older message history archives. Only one torrent at a time should be seeded.
Creating magnet links # Once a torrent file for all message archives is created, the control node MUST derive a magnet link following the Magnet URI scheme using the underlying BitTorrent protocol client.
Message archive distribution # Message archives are available via the BitTorrent network as they are being seeded by the control node. Other community member nodes will download the message archives from the BitTorrent network once they receive a magnet link that contains a message archive index.
The control node MUST send magnet links containing message archives and the message archive index to a special community channel. The topic of that special channel follows the following format:
/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding} All messages sent with this topic MUST be instances of ApplicationMetadataMessage (62/PAYLOADS) with a payload of CommunityMessageArchiveIndex.
Only the control node MAY post to the special channel. Other messages on this specified channel MUST be ignored by clients. Community members MUST NOT have permission to send messages to the special channel. However, community member nodes MUST subscribe to special channel to receive Waku messages containing magnet links for message archives.
Canonical message histories # Only control nodes are allowed to distribute messages with magnet links via the special channel for magnet link exchange. Community members MUST NOT be allowed to post any messages to the special channel.
Status nodes MUST ensure that any message that isn\u0026rsquo;t signed by the control node in the special channel is ignored.
Since the magnet links are created from the control node\u0026rsquo;s database (and previously distributed archives), the message history provided by the control node becomes the canonical message history and single source of truth for the community.
Community member nodes MUST replace messages in their local databases with the messages extracted from archives within the same time range. Messages that the control node didn\u0026rsquo;t receive MUST be removed and are no longer part of the message history of interest, even if it already existed in a community member node\u0026rsquo;s database.
Fetching message history archives # Generally, fetching message history archives is a three step process:
Receive message archive index magnet link as described in [Message archive distribution], download index file from torrent, then determine which message archives to download Download individual archives Community member nodes subscribe to the special channel that control nodes publish magnet links for message history archives to. There are two scenarios in which member nodes can receive such a magnet link message from the special channel:
The member node receives it via live messages, by listening to the special channel The member node requests messages for a time range of up to 30 days from store nodes (this is the case when a new community member joins a community) Downloading message archives # When member nodes receive a message with a CommunityMessageHistoryArchive (62/PAYLOADS) from the aforementioned channnel, they MUST extract the magnet_uri and pass it to their underlying BitTorrent client so they can fetch the latest message history archive index, which is the index file of the torrent (see Creating message archive torrents).
Due to the nature of distributed systems, there\u0026rsquo;s no guarantee that a received message is the \u0026ldquo;last\u0026rdquo; message. This is especially true when member nodes request historical messages from store nodes.
Therefore, member nodes MUST wait for 20 seconds after receiving the last CommunityMessageArchive before they start extracting the magnet link to fetch the latest archive index.
Once a message history archive index is downloaded and parsed back into WakuMessageArchiveIndex, community member nodes use a local lookup table to determine which of the listed archives are missing using the KECCAK-256 hashes stored in the index.
For this lookup to work, member nodes MUST store the KECCAK-256 hashes of the WakuMessageArchiveIndexMetadata provided by the index file for all of the message history archives that have been downlaoded in their local database.
Given a WakuMessageArchiveIndex, member nodes can access individual WakuMessageArchiveIndexMetadata to download individual archives.
Community member nodes MUST choose one of the following options:
Download all archives - Request and download all data pieces for data provided by the torrent (this is the case for new community member nodes that haven\u0026rsquo;t downloaded any archives yet) Download only the latest archive - Request and download all pieces starting at the offset of the latest WakuMessageArchiveIndexMetadata (this the case for any member node that already has downloaded all previous history and is now interested in only the latst archive) Download specific archives - Look into from and to fields of every WakuMessageArchiveIndexMetadata and determine the pieces for archives of a specific time range (can be the case for member nodes that have recently joined the network and are only interested in a subset of the complete history) Storing historical messages # When message archives are fetched, community member nodes MUST unwrap the resulting WakuMessage instances into ApplicationMetadataMessage instances and store them in their local database. Community member nodes SHOULD NOT store the wrapped WakuMessage messages.
All message within the same time range MUST be replaced with the messages provided by the message history archive.
Community members nodes MUST ignore the expiration state of each archive message.
Considerations # The following are things to cosider when implementing this specification.
Control node honesty # This spec assumes that all control nodes are honest and behave according to the spec. Meaning they don\u0026rsquo;t inject their own messages into, or remove any messages from historic archives.
Bandwidth consumption # Community member nodes will download the latest archive they\u0026rsquo;ve received from the archive index, which includes messages from the last seven days. Assuming that community members nodes were online for that time range, they have already downloaded that message data and will now download an archive that contains the same.
This means there\u0026rsquo;s a possibility member nodes will download the same data at least twice.
Multiple community owners # It is possible for control nodes to export the private key of their owned community and pass it to other users so they become control nodes as well. This means, it\u0026rsquo;s possible for multiple control nodes to exist.
This might conflict with the assumption that the control node serves as a single source of thruth. Multiple control nodes can have different message histories.
Not only will multiple control nodes multiply the amount of archive index messages being distributed to the network, they might also contain different sets of magnet links and their corresponding hashes.
Even if just a single message is missing in one of the histories, the hashes presented in archive indices will look completely different, resulting in the community member node to download the corresponding archive (which might be identical to an archive that was already downloaded, except for that one message).
Copyright # Copyright and related rights waived via CC0.
References # 13/WAKU2-STORE BitTorrent 10/WAKU2 11/WAKU2-RELAY Magnet URI scheme forum discussion org channels UI feature spec Extensions for Peers to Send Metadata Files org channels spec 14/WAKU2-MESSAGE 62/PAYLOAD `}),e.add({id:51,href:"/spec/63/",title:"63/STATUS-Keycard-Usage",section:"Docs",content:` Terminology # Account: A valid BIP-32 compliant key. Multiaccount: An account from which multiple Accounts can be derived. Abstract # This specification describes how an application can use the Status Keycard to -
Create Multiaccounts Store Multiaccounts Use Multiaccounts for transaction or message signing Derive Accounts from Multiaccounts More documentation on the Status Keycard can be found here
Motivation # The Status Keycard is a hardware wallet that can be used to store and sign transactions. For the purpose of the Status App, this specification describes how the Keycard SHOULD be used to store and sign transactions.
Usage # Endpoints # 1. Initialize Keycard (/init-keycard) # To initialize the keycard for use with the application. The keycard is locked with a 6 digit pin.
Request wire format # { \u0026#34;pin\u0026#34;: 6_digit_pin } Response wire format # { \u0026#34;password\u0026#34;: password_to_unlock_keycard, \u0026#34;puk\u0026#34;: 12_digit_recovery_code, \u0026#34;pin\u0026#34;: provided_pin, } The keycard MUST be initialized before it can be used with the application. The application SHOULD provide a way to recover the keycard in case the pin is forgotten.
2. Get Application Info (/get-application-info) # To fetch if the keycard is ready to be used by the application.
Request wire format # The requester MAY add a pairing field to filter through the generated keys
{ \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt; OR null } Response wire format # If the keycard is not initialized yet # { \u0026#34;initialized?\u0026#34;: false } If the keycard is initialized # { \u0026#34;free-pairing-slots\u0026#34;: number, \u0026#34;app-version\u0026#34;: major_version.minor_version, \u0026#34;secure-channel-pub-key\u0026#34;: valid_bip32_key,, \u0026#34;key-uid\u0026#34;: unique_id_of_the_default_key, \u0026#34;instance-uid\u0026#34;: unique_instance_id, \u0026#34;paired?\u0026#34;: bool, \u0026#34;has-master-key?\u0026#34;: bool, \u0026#34;initialized?\u0026#34; true } 3. Pairing the Keycard to the Client device (/pair) # To establish a secure communication channel described here, the keycard and the client device need to be paired.
Request wire format # { \u0026#34;password\u0026#34;: password_to_unlock_keycard } Response wire format # \u0026#34;\u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt;\u0026#34; 4. Generate a new set of keys (/generate-and-load-keys) # To generate a new set of keys and load them onto the keycard.
Request wire format # { \u0026#34;mnemonic\u0026#34;: 12_word_mnemonic, \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt;, \u0026#34;pin\u0026#34;: 6_digit_pin } Response wire format # { \u0026#34;whisper-address\u0026#34;: 20_byte_whisper_compatible_address, \u0026#34;whisper-private-key\u0026#34;: whisper_private_key, \u0026#34;wallet-root-public-key\u0026#34;: 256_bit_wallet_root_public_key, \u0026#34;encryption-public-key\u0026#34;: 256_bit_encryption_public_key,, \u0026#34;wallet-root-address\u0026#34;: 20_byte_wallet_root_address, \u0026#34;whisper-public-key\u0026#34;: 256_bit_whisper_public_key, \u0026#34;address\u0026#34;: 20_byte_address, \u0026#34;wallet-address\u0026#34;: 20_byte_wallet_address,, \u0026#34;key-uid\u0026#34;: 64_byte_unique_key_id, \u0026#34;wallet-public-key\u0026#34;: 256_bit_wallet_public_key, \u0026#34;public-key\u0026#34;: 256_bit_public_key, \u0026#34;instance-uid\u0026#34;: 32_byte_unique_instance_id, } 5. Get a set of generated keys (/get-keys) # To fetch the keys that are currently loaded on the keycard.
Request wire format # { \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt;, \u0026#34;pin\u0026#34;: 6_digit_pin } Response wire format # { \u0026#34;whisper-address\u0026#34;: 20_byte_whisper_compatible_address, \u0026#34;whisper-private-key\u0026#34;: whisper_private_key, \u0026#34;wallet-root-public-key\u0026#34;: 256_bit_wallet_root_public_key, \u0026#34;encryption-public-key\u0026#34;: 256_bit_encryption_public_key, \u0026#34;wallet-root-address\u0026#34;: 20_byte_wallet_root_address, \u0026#34;whisper-public-key\u0026#34;: 256_bit_whisper_public_key, \u0026#34;address\u0026#34;: 20_byte_address, \u0026#34;wallet-address\u0026#34;: 20_byte_wallet_address, \u0026#34;key-uid\u0026#34;: 64_byte_unique_key_id, \u0026#34;wallet-public-key\u0026#34;: 256_bit_wallet_public_key, \u0026#34;public-key\u0026#34;: 256_bit_public_key, \u0026#34;instance-uid\u0026#34;: 32_byte_unique_instance_id, } 6. Sign a transaction (/sign) # To sign a transaction using the keycard, passing in the pairing information and the transaction to be signed.
Request wire format # { \u0026#34;hash\u0026#34;: 64_byte_hash_of_the_transaction, \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt;, \u0026#34;pin\u0026#34;: 6_digit_pin, \u0026#34;path\u0026#34;: bip32_path_to_the_key } Response wire format # \u0026lt;256_bit_signature\u0026gt; 7. Export a key (/export-key) # To export a key from the keycard, passing in the pairing information and the path to the key to be exported.
Request wire format # { \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt;, \u0026#34;pin\u0026#34;: 6_digit_pin, \u0026#34;path\u0026#34;: bip32_path_to_the_key } Response wire format # \u0026lt;256_bit_public_key\u0026gt; 8. Verify a pin (/verify-pin) # To verify the pin of the keycard.
Request wire format # { \u0026#34;pin\u0026#34;: 6_digit_pin } Response wire format # 1_digit_status_code Status code reference:
3: PIN is valid 9. Change the pin (/change-pin) # To change the pin of the keycard.
Request wire format # { \u0026#34;new-pin\u0026#34;: 6_digit_new_pin, \u0026#34;current-pin\u0026#34;: 6_digit_new_pin, \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt; } Response wire format # If the operation was successful # true If the operation was unsuccessful # false 10. Unblock the keycard (/unblock-pin) # If the Keycard is blocked due to too many incorrect pin attempts, it can be unblocked using the PUK.
Request wire format # { \u0026#34;puk\u0026#34;: 12_digit_recovery_code, \u0026#34;new-pin\u0026#34;: 6_digit_new_pin, \u0026#34;pairing\u0026#34;: \u0026lt;shared_secret\u0026gt;/\u0026lt;pairing_index\u0026gt;/\u0026lt;256_bit_salt\u0026gt; } Response wire format # If the operation was successful # true If the operation was unsuccessful # false Flows # Any application that uses the Status Keycard MAY implement the following flows according to the actions listed above.
1. A new user wants to use the Keycard with the application # The user initializes the Keycard using the /init-keycard endpoint. The user pairs the Keycard with the client device using the /pair endpoint. The user generates a new set of keys using the /generate-and-load-keys endpoint. The user can now use the Keycard to sign transactions using the /sign endpoint. 2. An existing user wants to use the Keycard with the application # The user pairs the Keycard with the client device using the /pair endpoint. The user can now use the Keycard to sign transactions using the /sign endpoint. 3. An existing user wants to use the Keycard with a new client device # The user pairs the Keycard with the new client device using the /pair endpoint. The user can now use the Keycard to sign transactions using the /sign endpoint. 4. An existing user wishes to verify the pin of the Keycard # The user verifies the pin of the Keycard using the /verify-pin endpoint. 5. An existing user wishes to change the pin of the Keycard # The user changes the pin of the Keycard using the /change-pin endpoint. 6. An existing user wishes to unblock the Keycard # The user unblocks the Keycard using the /unblock-pin endpoint. Security Considerations # Inherits the security considerations of Status Keycard
Privacy Considerations # Inherits the privacy considerations of Status Keycard
Copyright # Copyright and related rights waived via CC0.
References # BIP-32 specification Keycard documentation 16/Keycard-Usage `}),e.add({id:52,href:"/spec/64/",title:"64/WAKU2-NETWORK",section:"Docs",content:` Abstract # This RFC specifies an opinionated deployment of 10/WAKU2 protocols to form a coherent and shared decentralized messaging network that is open-access, useful for generalized messaging, privacy-preserving, scalable and accessible even to resource-restricted devices. We\u0026rsquo;ll refer to this opinionated deployment simply as the public Waku Network, the Waku Network or, if the context is clear, the network in the rest of this document.
Theory / Semantics # Routing protocol # The Waku Network is built on the 17/WAKU2-RLN-RELAY routing protocol, which in turn is an extension of 11/WAKU2-RELAY with spam protection measures.
Network shards # Traffic in the Waku Network is sharded into eight 17/WAKU2-RLN-RELAY pubsub topics. Each pubsub topic is named according to the static shard naming format defined in 51/WAKU2-RELAY-SHARDING with:
\u0026lt;cluster_id\u0026gt; set to 1 \u0026lt;shard_number\u0026gt; occupying the range 0 to 7. In other words, the Waku Network is a 17/WAKU2-RLN-RELAY network routed on the combination of the eight pubsub topics: /waku/2/rs/1/0 /waku/2/rs/1/1 ... /waku/2/rs/1/7 A node MUST use WAKU-METADATA protocol to identify the \u0026lt;cluster_id\u0026gt; that every inbound/outbound peer that attempts to connect supports. In any of the following cases, the node MUST trigger a disconnection:
WAKU-METADATA dial fails. WAKU-METADATA reports an empty \u0026lt;cluster_id\u0026gt;. WAKU-METADATA reports a \u0026lt;cluster_id\u0026gt; different than 1. Roles # There are two distinct roles evident in the network, those of:
nodes, and applications. Nodes # Nodes are the individual software units using 10/WAKU2 protocols to form a p2p messaging network. Nodes, in turn, can participate in a shard as full relayers, i.e. relay nodes, or by running a combination of protocols suitable for resource-restricted environments, i.e. non-relay nodes. Nodes can also provide various services to the network, such as storing historical messages or protecting the network against spam. See the section on default services for more.
Relay nodes # Relay nodes MUST follow 17/WAKU2-RLN-RELAY to route messages to other nodes in the network for any of the pubsub topics defined as the Waku Network shards. Relay nodes MAY choose to subscribe to any of these shards, but MUST be subscribed to at least one defined shard. Each relay node SHOULD be subscribed to as many shards as it has resources to support. If a relay node supports an encapsulating application, it SHOULD be subscribed to all the shards servicing that application. If resource restrictions prevent a relay node from servicing all shards used by the encapsulating application, it MAY choose to support some shards as a non-relay node.
Bootstrapping and discovery # Nodes MAY use any method to bootstrap connection to the network, but it is RECOMMENDED that each node retrieves a list of bootstrap peers to connect to using EIP-1459 DNS-based discovery. Relay nodes SHOULD use 33/WAKU2-DISCV5 to continually discover other peers in the network. Each relay node MUST encode its supported shards into its discoverable ENR as described in 51/WAKU2-RELAY-SHARDING. The ENR MUST be updated if the set of supported shards change. A node MAY choose to ignore discovered peers that do not support any of the shards in its own subscribed set.
Transports # Relay nodes MUST follow 10/WAKU2 specifications with regards to supporting different transports. If TCP transport is available, each relay node MUST support it as transport for both dialing and listening. In addition, a relay node SHOULD support secure websockets for bidirectional communication streams, for example to allow connections from and to web browser-based clients. A relay node MAY support unsecure websockets if required by the application or running environment.
Default services # For each supported shard, each relay node SHOULD enable and support the following protocols as a service node:
12/WAKU2-FILTER to allow resource-restricted peers to subscribe to messages matching a specific content filter. 13/WAKU2-STORE to allow other peers to request historical messages from this node. 19/WAKU2-LIGHTPUSH to allow resource-restricted peers to request publishing a message to the network on their behalf. 34/WAKU2-PEER-EXCHANGE to allow resource-restricted peers to discover more peers in a resource efficient way. Store service nodes # Each relay node SHOULD support 13/WAKU2-STORE as a store service node, for each supported shard. The store SHOULD be configured to retain at least 12 hours of messages per supported shard. Store service nodes SHOULD only store messages with a valid rate_limit_proof attribute.
Non-relay nodes # Nodes MAY opt out of relay functionality on any network shard and instead request services from relay nodes as clients using any of the defined service protocols:
12/WAKU2-FILTER to subscribe to messages matching a specific content filter. 13/WAKU2-STORE to request historical messages matching a specific content filter. 19/WAKU2-LIGHTPUSH to request publishing a message to the network. 34/WAKU2-PEER-EXCHANGE to discover more peers in a resource efficient way. Store client nodes # Nodes MAY request historical messages from 13/WAKU2-STORE service nodes as store clients. A store client SHOULD discard any messages retrieved from a store service node that do not contain a valid rate_limit_proof attribute. The client MAY consider service nodes returning messages without a valid rate_limit_proof attribute as untrustworthy. The mechanism by which this may happen is currently underdefined.
Applications # Applications are the higher-layer projects or platforms that make use of the generalized messaging capability of the network. In other words, an application defines a payload used in the various 10/WAKU2 protocols. Any participant in an application SHOULD make use of an underlying node in order to communicate on the network. Applications SHOULD make use of an autosharding API to allow the underlying node to automatically select the target shard on the Waku Network. See the section on autosharding for more.
RLN rate-limiting # The 17/WAKU2-RLN-RELAY network uses 32/RLN-V1 proofs to ensure that a pre-agreed rate limit is not exceeded by any publisher. While the network is under capacity, individual relayers MAY choose to freely route messages without RLN proofs up to a discretionary bandwidth limit after which messages without proofs MUST be discarded. This bandwidth limit SHOULD be enforced using bandwidth validation mechanism separate from RLN rate-limiting. This implies that quality of service and reliability is significantly lower for messages without proofs and at times of high network utilization these messages may not be relayed at all.
RLN Parameters # For the Waku Network, the epoch is set to 1 second and the maximum number of messages published per epoch is limited to 1 per publisher. The max_epoch_gap is set to 20 seconds, meaning that validators MUST reject messages with an epoch more than 20 seconds into the past or future compared to the validator\u0026rsquo;s own clock. All nodes, validators and publishers, SHOULD use Network Time Protocol (NTP) to synchronize their own clocks, thereby ensuring valid timestamps for proof generation and validation.
Memberships # Each publisher to the Waku Network SHOULD register an RLN membership with one of the RLN storage contracts moderated in the Sepolia registry contract with address 0xF1935b338321013f11068abCafC548A7B0db732C. Initial memberships are registered in the Sepolia RLN storage contract with address 0x58322513A35a8f747AF5A385bA14C2AbE602AA59. RLN membership setup and registration MUST follow 17/WAKU-RLN-RELAY, with the staked_fund set to 0. In other words, the Waku Network does not use RLN staking.
RLN Proofs # Each RLN member MUST generate and attach an RLN proof to every published message as described in 17/WAKU-RLN-RELAY. Slashing is not implemented for the Waku Network. Instead, validators will penalise peers forwarding messages exceeding the rate limit as specified for the rate-limiting validation mechanism. This incentivizes all nodes to validate RLN proofs and reject messages violating rate limits in order to continue participating in the network.
Network traffic # All payload on the Waku Network MUST be encapsulated in a 14/WAKU2-MESSAGE with rate limit proof extensions defined for 17/WAKU2-RLN-RELAY. Each message on the Waku Network SHOULD be validated by each relayer, according to the rules discussed under message validation.
Message Attributes # The mandatory payload attribute MUST contain the message data payload as crafted by the application. The mandatory content_topic attribute MUST specify a string identifier that can be used for content-based filtering. This is also crafted by the application. See Autosharding for more on the content topic format. The optional meta attribute MAY be omitted. If present this will form part of the message uniqueness vector described in 14/WAKU2-MESSAGE. The optional version attribute SHOULD be set to 0. It MUST be interpreted as 0 if not present. The mandatory timestamp attribute MUST contain the Unix epoch time at which the message was generated by the application. The value MUST be in nanoseconds. It MAY contain a fudge factor of up to 1 seconds in either direction to improve resistance to timing attacks. The optional ephemeral attribute MUST be set to true if the message should not be persisted by the Waku Network. The optional rate_limit_proof attribute SHOULD be populated with the RLN proof as set out in RLN Proofs. Messages with this field unpopulated MAY be discarded from the network by relayers. This field MUST be populated if the message should be persisted by the Waku Network. Message Size # Any Waku Message published to the network MUST NOT exceed an absolute maximum size of 150 kilobytes. This limit applies to the entire message after protobuf serialization, including attributes. It is RECOMMENDED not to exceed an average size of 4 kilobytes for Waku Messages published to the network.
Message Validation # Relay nodes MUST apply gossipsub v1.1 validation to each relayed message and SHOULD apply all of the rules set out in the section below to determine the validity of a message. Validation has one of three outcomes, repeated here from the gossipsub specification for ease of reference:
Accept - the message is considered valid and it MUST be delivered and forwarded to the network. Reject - the message is considered invalid, MUST be rejected and SHOULD trigger a gossipsub scoring penalty against the transmitting peer. Ignore - the message SHOULD NOT be delivered and forwarded to the network, but this MUST NOT trigger a gossipsub scoring penalty against the transmitting peer. The following validation rules are defined:
Decoding failure # If a message fails to decode as a valid 14/WAKU2-MESSAGE. the relay node MUST reject the message. This SHOULD trigger a penalty against the transmitting peer.
Invalid timestamp # If a message has a timestamp deviating by more than 20 seconds either into the past or the future when compared to the relay node\u0026rsquo;s internal clock, the relay node MUST reject the message. This allows for some deviation between internal clocks, network routing latency and an optional fudge factor when timestamping new messages.
Free bandwidth exceeded # If a message contains no RLN proof and the current bandwidth utilization on the shard the message was published to equals or exceeds 1 Mbps, the relay node SHOULD ignore the message.
Invalid RLN epoch # If a message contains an RLN proof and the epoch attached to the proof deviates by more than max_epoch_gap seconds from the relay node\u0026rsquo;s own epoch, the relay node MUST reject the message. max_epoch_gap is set to 20 seconds for the Waku Network.
Invalid RLN proof # If a message contains an RLN proof and the zero-knowledge proof is invalid according to the verification process described in 32/RLN-V1, the relay node MUST ignore the message.
Rate limit exceeded # If a message contains an RLN proof and the relay node detects double signaling according to the verification process described in 32/RLN-V1, the relay node MUST reject the message for violating the agreed rate limit of 1 message every 1 second. This SHOULD trigger a penalty against the transmitting peer.
Autosharding # Nodes in the Waku Network SHOULD allow encapsulating applications to use autosharding, as defined in 51/WAKU2-RELAY-SHARDING by automatically determining the appropriate pubsub topic from the list of defined Waku Network shards. This allows the application to omit the target pubsub topic when invoking any Waku protocol function. Applications using autosharding MUST use content topics in the format defined in 51/WAKU2-RELAY-SHARDING and SHOULD use the short length format:
/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}\` When an encapsulating application makes use of autosharding the underlying node MUST determine the target pubsub topic(s) from the content topics provided by the application using the hashing mechanism defined in 51/WAKU2-RELAY-SHARDING.
Copyright # Copyright and related rights waived via CC0.
References # (TBD)
normative # (TBD) A list of references that MUST be read to fully understand and/or implement this protocol. See RFC3967 Section 1.1.
informative # (TBD) A list of additional references.
`}),e.add({id:53,href:"/spec/65/",title:"65/STATUS-ACCOUNTS",section:"Docs",content:` Abstract # This specification explains what a Status account is, and how it is created and used.
Background # The core concept of an account in Status is a set of cryptographic keypairs. Namely, the combination of the following:
a Waku chat identity keypair a set of cryptocurrency wallet keypairs The Status node verifies or derives everything else associated with the contact from the above items, including:
Ethereum address (future verification, currently the same base keypair) identicon message signatures Initial Key Generation # Public/Private Keypairs # An ECDSA (secp256k1 curve) public/private keypair MUST be generated via a BIP43 derived path from a BIP39 mnemonic seed phrase. The default paths are defined as such: Waku Chat Key (IK): m/43'/60'/1581'/0'/0 (post Multiaccount integration) following EIP1581 Status Wallet paths: m/44'/60'/0'/0/i starting at i=0 following BIP44 NOTE: this (i=0) is also the current (and only) path for Waku key before Multiaccount integration Account Broadcasting # A user is responsible for broadcasting certain information publicly so that others may contact them. X3DH Prekey bundles # Refer to 53/WAKU2-X3DH for details on the X3DH prekey bundle broadcasting, as well as regeneration. Optional Account additions # ENS Username # A user MAY register a public username on the Ethereum Name System (ENS). This username is a user-chosen subdomain of the stateofus.eth ENS registration that maps to their Waku identity key (IK). User Profile Picture # An account MAY edit the IK generated identicon with a chosen picture. This picture will become part of the publicly broadcasted profile of the account. Wire Format # Below is the wire format for the account information that is broadcasted publicly. An Account is referred to as a Multiaccount in the wire format.
message MultiAccount { string name = 1; // name of the account int64 timestamp = 2; // timestamp of the message string identicon = 3; // base64 encoded identicon repeated ColorHash color_hash = 4; // color hash of the identicon int64 color_id = 5; // color id of the identicon string keycard_pairing = 6; // keycard pairing code string key_uid = 7; // unique identifier of the account repeated IdentityImage images = 8; // images associated with the account string customization_color = 9; // color of the identicon uint64 customization_color_clock = 10; // clock of the identicon color, to track updates message ColorHash { repeated int64 index = 1; } message IdentityImage { string key_uid = 1; // unique identifier of the image string name = 2; // name of the image bytes payload = 3; // payload of the image int64 width = 4; // width of the image int64 height = 5; // height of the image int64 filesize = 6; // filesize of the image int64 resize_target = 7; // resize target of the image uint64 clock = 8; // clock of the image, to track updates } } The above payload is broadcasted when 2 devices that belong to a user need to be paired.
Security Considerations # This specification inherits security considerations of 53/WAKU2-X3DH and 54/WAKU2-X3DH-SESSIONS. Copyright # Copyright and related rights waived via CC0.
References # normative # 53/WAKU2-X3DH 54/WAKU2-X3DH-SESSIONS 55/STATUS-1TO1-CHAT informative # BIP43 BIP39 EIP1581 BIP44 Ethereum Name System Status Multiaccount `}),e.add({id:54,href:"/spec/66/",title:"66/WAKU2-METADATA",section:"Docs",content:` Metadata Protocol # Waku specifies a req/resp protocol that provides information about the node\u0026rsquo;s medatadata. Such metadata is meant to be used by the node to decide if a peer is worth connecting or not. The node that makes the request, includes its metadata so that the receiver is aware of it, without requiring an extra interaction. The parameters are the following:
clusterId: Unique identifier of the cluster that the node is running in. shards: Shard indexes that the node is subscribed to. Protocol id # /vac/waku/metadata/1.0.0
Request # message WakuMetadataRequest { optional uint32 cluster_id = 1; repeated uint32 shards = 2; } Response # message WakuMetadataResponse { optional uint32 cluster_id = 1; repeated uint32 shards = 2; } `}),e.add({id:55,href:"/spec/7/",title:"7/WAKU-DATA",section:"Docs",content:`This specification describes the encryption, decryption and signing of the content in the data field used in Waku.
Specification # The data field is used within the waku envelope, the field MUST contain the encrypted payload of the envelope.
The fields that are concatenated and encrypted as part of the data field are:
flags auxiliary field payload padding signature In case of symmetric encryption, a salt (a.k.a. AES Nonce, 12 bytes) field MUST be appended.
ABNF # Using Augmented Backus-Naur form (ABNF) we have the following format:
; 1 byte; first two bits contain the size of auxiliary field, ; third bit indicates whether the signature is present. flags = 1OCTET ; contains the size of payload. auxiliary-field = 4*OCTET ; byte array of arbitrary size (may be zero) payload = *OCTET ; byte array of arbitrary size (may be zero). padding = *OCTET ; 65 bytes, if present. signature = 65OCTET ; 2 bytes, if present (in case of symmetric encryption). salt = 2OCTET data = flags auxiliary-field payload padding [signature] [salt] Signature # Those unable to decrypt the envelope data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialized as the concatenation of the R, S and V parameters of the SECP-256k1 ECDSA signature, in that order. R and S MUST be big-endian encoded, fixed-width 256-bit unsigned. V MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
Padding # The padding field is used to align data size, since data size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the Salt) before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:56,href:"/spec/70/",title:"70/ETH-SECPM",section:"Docs",content:` Abstract # This document specifies an Ethereum-based private messaging service. This proposal is built upon this model and amends the limitations of the latter concerning forward privacy and authentication. The document is still work in progress. Next steps will include a description of how to implement the different functions and algorithms in terms of the Noise framework.
Background # Alice wants to send an encrypted message to Bob. Here Bob is the only individual able to decrypt the message. Alice has access to Bob’s Ethereum address.
Theory and Description of the Protocol # The proposed protocol must adhere to the following design requirements:
Alice knows Bob’s Ethereum address. Bob is willing to participate in the protocol, and publishes his public key. Bob’s ownership of his public key is verifiable, Alice wants to send message M to Bob. An eavesdropper cannot read M’s content even if she is storing it or relaying it. The specification is based on the noise protocol framework. It corresponds to the double ratchet scheme combined with the X3DH algorithm, which will be used to initialize the former. We chose to express the protocol in noise to be be able to use the noise streamlined implementation and proving features. The X3DH algorithm provides both authentication and forward secrecy, as stated in the X3DH specification.
High level description # This protocol will consist of several stages:
Key setting for X3DH: this step will produce prekey bundles for Bob which will be fed into X3DH. It will also allow Alice to generate the keys required to run the X3DH algorithm correctly. Execution of X3DH: This step will output a common secret key SK together with an additional data vector AD. Both will be used in the Double Ratchet algorithm initialization. Execution of the Double Ratchet algorithm for forward secure, authenticated communications, using the common secret key SK, obtained from X3DH, as a root key. Cryptographic functions required # XEd448 for digital signatures involved in the X3DH key generation. SHA512 for hashing and the generation of HMACs. AES256-CBC for the encryption/decryption of messages. Considerations on the X3DH initialization # This scheme requires working on specific elliptic curves which differ from those used by Ethereum. To be precise, Ethereum makes use of the curve secp256k1, whereas X3DH requires either X25519 or X448. For security reasons one must work on the curve X448.
Bob and Alice must define a key pair (ik, IK) where:
The key ik must be kept secret, and the key IK is public. Bob will not be able to use his Ethereum public key during this stage due to incompatibilities with the involved elliptic curves, therefore it will be required to generate new keys. This can be done using the basepoint $G$ for X448 and $ik \\in \\mathbb{Z}_p$ a random integer:
$$ IK = ik \\cdot G $$
The scheme X3DH will also require the generation of a public key SPK which will be generated repeating the above process: one takes $spk \\in \\mathbb{Z}_p$ a secret random integer and computes:
$$ SPK = spk \\cdot G $$
SPK is a public key generated and stored at medium-term. It is called a signed prekey because Bob also needs to store a public key certificate of SPK using IK. Both signed prekey and the certificate must undergo periodic replacement, a process that entails the generation of a fresh signed prekey. After replacing the key, Bob keeps the old private key of SPK for some interval, dependant on the implementation. This allows Bob to decrypt delayed messages. It is important that Bob does not reuse SPKs. This action is pivotal for ensuring forward secrecy, as these keys are integral for recalculating the shared secret employed in decrypting historical messages.
It will be required to sign SPK for authentication. Following the specification of X3DH, one will use the digital signature scheme XEd448 and define:
$$ SigSPK = XEd448(ik, Encode(SPK)) $$
A final step requires the definition of a prekey bundle given by the tuple
$$ prekey_bundle = (IK, SPK, SigSPK, {OPK_i}_i) $$
Where the different one-time keys OPK are points in X448 generated from a random integer $opk \\in \\mathbb{Z}_p$ and computed by performing
$$ OPK = opk\\cdot G $$
Before sending an initial message to Bob, Alice will generate an AD vector as described in the documentation:
$$ AD = Encode(IK_A)|| Encode(IK_B) $$
Alice will also need to generate ephemeral key pairs (ek, EK) following the above mechanisms, that is: ek is a random integer modulo p, and EK is the associated public key obtained from the product
$$ EK = ek \\cdot G $$
The function Encode() transforms an X448 public key into a byte sequence. The recommended encoding consists of a single-byte constant to represent the type of curve, followed by little-endian encoding of the u-coordinate. This is specified in the RFC 7748 on elliptic curves for security.
Using X3DH in Double Ratchet # According to Signal specifications this specification uses the double ratchet in combination with X3DH using the following data as initialization for the former:
The SK output from X3DH becomes the SK input of the double ratchet. See section 3.3 of Signal Specification for a detailed description. The AD output from X3DH becomes the AD input of the double ratchet. See sections 3.4 and 3.5 of Signal Specification for a detailed description. Bob’s signed prekey SigSPKB from X3DH is used as Bob’s initial ratchet public key of the double ratchet. Once this initialization has been set, Alice and Bob can start exchanging messages with forward secrecy and authentication.
Specification as a Noise protocol # X3DH has three phases:
Bob publishes his identity key and prekeys to a server, or dedicated smart contract. Alice fetches a \u0026ldquo;prekey bundle\u0026rdquo; from the server, and uses it to send an initial message to Bob. Bob receives and processes Alice\u0026rsquo;s initial message. One observes that, at the beginning of the protocol, the receiver gets the public key through a server, a smart contract in our situation, together with an encrypted ephemeral key. This corresponds to the Noise pattern IX:
→ e, s ← e, s, es, se, ee
The Diffie-Hellman ratchet is run using the valid private key of the receiver in combination with the valid public included in the message coming from the sender. This process is encoded, in Noise terms, as the DH() function. This function will have inputs the secret key of the user running the function, and the public key of the external user. Receiver and sender MUST generate valid key pairs, i.e. points of the X448, using the Noise function GENERATE_KEYPAIR().
The Key Derivation Function (KDF) ratchet and the associated encryption protocols used by the double ratchet are also included by the Noise framework: SHA256 for the KDF and AES256 for AEAD encryption.
Consequently, according to the Noise framework specifications, the X3DH algorithm is encoded as Noise_IX_448_AES256GCM_SHA256
Retrieving information # Static data # Some data, such as the key pairs (ik, IK) for Alice and Bob, do not need to be regenerated after a period of time. Therefore the public keys IK can be stored in long-term storage solutions, such as a dedicated smart contract which outputs such a key pair when receiving an Ethereum wallet address.
Ephemeral data # Storing ephemeral data on Ethereum can be done using a combination of on-chain and off-chain solutions. This approach provides an efficient solution to the problem of storing updatable data in Ethereum.
Ethereum can store a reference or a hash that points to the off-chain data. Off-chain solutions can include systems like IPFS, traditional cloud storage solutions, or decentralized storage networks such as a Swarm. In any case, the user stores the associated IPFS hash, URL or reference in Ethereum. The fact of a user not updating the ephemeral information can be understood as Bob not willing to participate in any communication.
Interaction with Ethereum # Storing static data is done using a dedicated smart contract PublicKeyStorage which associates the Ethereum wallet address of a user with his public key. This mapping is done by PublicKeyStorage using a publicKeys function, or a setPublicKey function. This mapping is done if the user passed an authorization process. A user who wants to retrieve a public key associated with a specific wallet address calls a function getPublicKey. The user provides the wallet address as the only input parameter for getPublicKey. The function outputs the associated public key from the smart contract.
Extension to group chat # 1-to-1 version # In order to extend the protocol to a group chat, this document specifies using an Asynchronous Distributed Key Generation (ADKG) to replace the X3DH step in the previous combination X3DH + Double Ratchet.
Distributed Key Generation (DKG) is a method for initiating threshold cryptosystems in a decentralized manner, all without the need for a trusted third party. DKG serves as a fundamental component for numerous decentralized protocols, including systems like randomness beacons, threshold signatures, Byzantine consensus, and multiparty computation.
Most DKG protocols assume synchronous networks. Asynchronous DKG (ADKG) has been studied only recently and the state-of-the-art high-threshold ADKG protocols is very inefficient compared to its low-threshold counterpart.
Here low-threshold means that the reconstruction threshold is set to be one higher than the number of corrupt nodes, whereas high-threshold protocols admit reconstruction thresholds much higher than the number of malicious nodes.
Existing ADKG constructions tend to become inefficient when the reconstruction threshold surpasses one-third of the total nodes. In this proposal we suggest using the scheme by Kokoris-Kogias et al. which is designed for $n = 3t + 1$ nodes.
This protocol can withstand the presence of up to t malicious nodes and can adapt to any reconstruction threshold in $l \\in [t, n-t-1]$. The key point of the proposal is an asynchronous method for securely distributing a random polynomial of degree $l\\geq t$. The proposal includes Python and Rust implementations.
The DKG suggested makes assumes the existence of a PKI. In case of requiring removing such assumption, one can replace the VSS scheme with the Alhaddad et al. at the price of increasing the complexity.
The output of the DKG may be an integer (modulo a prime), meaning that one should apply a KDF to that output in order to obtain a result which could be used as an input for the double ratchet.
One observes that using an ADKG allows a set of users, which want to define a group chat, defining a common secret key which will be used as a root key for the double ratchet. Using an ADKG defines a room key, which essentially defines the group itself.
This approach share similarities with the point of view of Farcaster.
Once the double ratchet is initialized, the communication in this group is 1-to-1, meaning that group member C cannot see the messages between group members A and B. The fact of defining a room key makes impossible for outsiders to communicate with group members if the latter are not willing to.
n-to-n version # Using the above approach leads to a situation where a group of users can set a group for 1-to-1 messages, meaning that any group member external to a communication between any other two members will not be able to read the contents of the messages.
An approach to generalize this situation to the setting of a group of users exchanging messages without any kind of restriction is using asynchronous ratcheting trees, as suggested in the proposal from Cohn-Gordon et al. where a group of people can derive a shared secret key even in the event of if no two users are ever online at the same time. The proposal suggested provides both forward secrecy and post-compromise security. The shared key can be then used in any symmetric encryption scheme, such as AES256.
Privacy and Security Considerations # For the information retrieval, the algorithm MUST include a access control mechanisms to restrict who can call the set and get functions. One SHOULD include event logs to track changes in public keys. The curve X448 MUST be chosen as the elliptic curve, since it offers a higher security level: 224-bit security instead of the 128-bit security provided by X25519. Concerning the hardness of the ADKG, the proposal lies on the Discrete Logarithm assumption. Copyright # Copyright and related rights waived via CC0.
References # https://rfc.vac.dev/spec/20/ https://signal.org/docs/specifications/x3dh/ https://signal.org/docs/specifications/doubleratchet/ https://eprint.iacr.org/2022/1389 https://github.com/sourav1547/htadkg https://github.com/farcasterxyz/protocol/discussions/99 `}),e.add({id:57,href:"/spec/8/",title:"8/WAKU-MAIL",section:"Docs",content:` Abstract # In this specification, we describe Mailservers. These are nodes responsible for archiving envelopes and delivering them to peers on-demand.
Specification # A node which wants to provide mailserver functionality MUST store envelopes from incoming Messages packets (Waku packet-code 0x01). The envelopes can be stored in any format, however they MUST be serialized and deserialized to the Waku envelope format.
A mailserver SHOULD store envelopes for all topics to be generally useful for any peer, however for specific use cases it MAY store envelopes for a subset of topics.
Requesting Historic Envelopes # In order to request historic envelopes, a node MUST send a packet P2P Request (0x7e) to a peer providing mailserver functionality. This packet requires one argument which MUST be a Waku envelope.
In the Waku envelope\u0026rsquo;s payload section, there MUST be RLP-encoded information about the details of the request:
; UNIX time in seconds; oldest requested envelope\u0026#39;s creation time lower = 4OCTET ; UNIX time in seconds; newest requested envelope\u0026#39;s creation time upper = 4OCTET ; array of Waku topics encoded in a bloom filter to filter envelopes bloom = 64OCTET ; unsigned integer limiting the number of returned envelopes limit = 4OCTET ; array of a cursor returned from the previous request (optional) cursor = *OCTET ; List of topics interested in topics = \u0026#34;[\u0026#34; *1000topic \u0026#34;]\u0026#34; ; 4 bytes of arbitrary data topic = 4OCTET payload-without-topic = \u0026#34;[\u0026#34; lower upper bloom limit [ cursor ] \u0026#34;]\u0026#34; payload-with-topic = \u0026#34;[\u0026#34; lower upper bloom limit cursor [ topics ] \u0026#34;]\u0026#34; payload = payload-with-topic | payload-without-topic The Cursor field SHOULD be filled in if a number of envelopes between Lower and Upper is greater than Limit so that the requester can send another request using the obtained Cursor value. What exactly is in the Cursor is up to the implementation. The requester SHOULD NOT use a Cursor obtained from one mailserver in a request to another mailserver because the format or the result MAY be different.
The envelope MUST be encrypted with a symmetric key agreed between the requester and Mailserver.
If Topics is used the Cursor field MUST be specified for the argument order to be unambiguous. However, it MAY be set to null. Topics is used to specify which topics a node is interested in. If Topics is not empty, a mailserver MUST only send envelopes that belong to a topic from Topics list and Bloom value MUST be ignored.
Receiving Historic Envelopes # Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (0x7f) followed by an array of Waku envelopes. A Mailserver MUST limit the amount of messages sent, either by the Limit specified in the request or limited to the maximum RLPx packet size, whichever limit comes first.
In order to receive historic envelopes from a mailserver, a node MUST trust the selected mailserver, that is allow to receive expired packets with the P2P Message code. By default, such packets are discarded.
Received envelopes MUST be passed through the Whisper envelope pipelines so that they are picked up by registered filters and passed to subscribers.
For a requester, to know that all envelopes have been sent by mailserver, it SHOULD handle P2P Request Complete code (0x7d). This code is followed by a list with:
; array with a Keccak-256 hash of the envelope containing the original request. request-id = 32OCTET ; array with a Keccak-256 hash of the last sent envelope for the request. last-envelope-hash = 32OCTET ; array of a cursor returned from the previous request (optional) cursor = *OCTET payload = \u0026#34;[\u0026#34; request-id last-envelope-hash [ cursor ] \u0026#34;]\u0026#34; If Cursor is not empty, it means that not all envelopes were sent due to the set Limit in the request. One or more consecutive requests MAY be sent with Cursor field filled in in order to receive the rest of the envelopes.
Security considerations # There are several security considerations to take into account when running or interacting with Mailservers. Chief among them are: scalability, DDoS-resistance and privacy.
Mailserver High Availability requirement:
A mailserver has to be online to receive envelopes for other nodes, this puts a high availability requirement on it.
Mailserver client privacy:
A mailserver client fetches archival envelopes from a mailserver through a direct connection. In this direct connection, the client discloses its IP/ID as well as the topics/ bloom filter it is interested in to the mailserver. The collection of such information allows the mailserver to link clients\u0026rsquo; IP/IDs to their topic interests and build a profile for each client over time. As such, the mailserver client has to trust the mailserver with this level of information. A similar concern exists for the light nodes and their direct peers which is discussed in the security considerations of 6/WAKU1.
Mailserver trusted connection:
A mailserver has a direct TCP connection, which means they are trusted to send traffic. This means a malicious or malfunctioning mailserver can overwhelm an individual node.
Changelog # Version Comment 1.0.0 marked stable as it is in use. 0.2.0 Add topic interest to reduce bandwidth usage 0.1.0 Initial Release Difference between wms 0.1 and wms 0.2 # topics option Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:58,href:"/spec/9/",title:"9/WAKU-RPC",section:"Docs",content:`This specification describes the RPC API that Waku nodes MAY adhere to. The unified API allows clients to easily be able to connect to any node implementation. The API described is privileged as a node stores the keys of clients.
Introduction # This API is based off the Whisper V6 RPC API.
Wire Protocol # Transport # Nodes SHOULD expose a JSON RPC API that can be accessed. The JSON RPC version SHOULD be 2.0. Below is an example request:
{ \u0026#34;jsonrpc\u0026#34;:\u0026#34;2.0\u0026#34;, \u0026#34;method\u0026#34;:\u0026#34;waku_version\u0026#34;, \u0026#34;params\u0026#34;:[], \u0026#34;id\u0026#34;:1 } Fields # Field Description jsonrpc Contains the used JSON RPC version (Default: 2.0) method Contains the JSON RPC method that is being called params An array of parameters for the request id The request ID Objects # In this section you will find objects used throughout the JSON RPC API.
Message # The message object represents a Waku message. Below you will find the description of the attributes contained in the message object. A message is the decrypted payload and padding of an envelope along with all of its metadata and other extra information such as the hash.
Field Type Description sig string Public Key that signed this message recipientPublicKey string The recipients public key ttl number Time-to-live in seconds timestamp number Unix timestamp of the message generation topic string 4 bytes, the message topic payload string Decrypted payload padding string Optional padding, byte array of arbitrary length pow number The proof of work value hash string Hash of the enveloped message Filter # The filter object represents filters that can be applied to retrieve messages. Below you will find the description of the attributes contained in the filter object.
Field Type Description symKeyID string ID of the symmetric key for message decryption privateKeyID string ID of private (asymmetric) key for message decryption sig string Public key of the signature minPow number Minimal PoW requirement for incoming messages topics array Array of possible topics, this can also contain partial topics allowP2P boolean Indicates if this filter allows processing of direct peer-to-peer messages All fields are optional, however symKeyID or privateKeyID must be present, it cannot be both. Additionally, the topics field is only optional when an asymmetric key is used.
Methods # waku_version # The waku_version method returns the current version number.
Parameters # none
Response # string - The version number. waku_info # The waku_info method returns information about a Waku node.
Parameters # none
Response # The response is an Object containing the following fields:
minPow [number] - The current PoW requirement. maxEnvelopeSize [float] - The current maximum envelope size in bytes. memory [number] - The memory size of the floating messages in bytes. envelopes [number] - The number of floating envelopes. waku_setMaxEnvelopeSize # Sets the maximum envelope size allowed by this node. Any envelopes larger than this size both incoming and outgoing will be rejected. The envelope size can never exceed the underlying envelope size of 10mb.
Parameters # number - The message size in bytes. Response # bool - true on success or an error on failure. waku_setMinPoW # Sets the minimal PoW required by this node.
Parameters # number - The new PoW requirement. Response # bool - true on success or an error on failure. waku_markTrustedPeer # Marks a specific peer as trusted allowing it to send expired messages.
Parameters # string - enode of the peer. Response # bool - true on success or an error on failure. waku_newKeyPair # Generates a keypair used for message encryption and decryption.
Parameters # none
Response # string - Key ID on success or an error on failure. waku_addPrivateKey # Stores a key and returns its ID.
Parameters # string - Private key as hex bytes. Response # string - Key ID on success or an error on failure. waku_deleteKeyPair # Deletes a specific key if it exists.
Parameters # string - ID of the Key pair. Response # bool - true on success or an error on failure. waku_hasKeyPair # Checks if the node has a private key of a key pair matching the given ID.
Parameters # string - ID of the Key pair. Response # bool - true or false or an error on failure. waku_getPublicKey # Returns the public key for an ID.
Parameters # string - ID of the Key. Response # string - The public key or an error on failure. waku_getPrivateKey # Returns the private key for an ID.
Parameters # string - ID of the Key. Response # string - The private key or an error on failure. waku_newSymKey # Generates a random symmetric key and stores it under an ID. This key can be used to encrypt and decrypt messages where the key is known to both parties.
Parameters # none
Response # string - The key ID or an error on failure. waku_addSymKey # Stores the key and returns its ID.
Parameters # string - The raw key for symmetric encryption hex encoded. Response # string - The key ID or an error on failure. waku_generateSymKeyFromPassword # Generates the key from a password and stores it.
Parameters # string - The password. Response # string - The key ID or an error on failure. waku_hasSymKey # Returns whether there is a key associated with the ID.
Parameters # string - ID of the Key. Response # bool - true or false or an error on failure. waku_getSymKey # Returns the symmetric key associated with an ID.
Parameters # string - ID of the Key. Response # string - Raw key on success or an error of failure. waku_deleteSymKey # Deletes the key associated with an ID.
Parameters # string - ID of the Key. Response # bool - true or false or an error on failure. waku_subscribe # Creates and registers a new subscription to receive notifications for inbound Waku messages.
Parameters # The parameters for this request is an array containing the following fields:
string - The ID of the function call, in case of Waku this must contain the value \u0026ldquo;messages\u0026rdquo;. object - The message filter. Response # string - ID of the subscription or an error on failure. Notifications # Notifications received by the client contain a message matching the filter. Below is an example notification:
{ \u0026#34;jsonrpc\u0026#34;: \u0026#34;2.0\u0026#34;, \u0026#34;method\u0026#34;: \u0026#34;waku_subscription\u0026#34;, \u0026#34;params\u0026#34;: { \u0026#34;subscription\u0026#34;: \u0026#34;02c1f5c953804acee3b68eda6c0afe3f1b4e0bec73c7445e10d45da333616412\u0026#34;, \u0026#34;result\u0026#34;: { \u0026#34;sig\u0026#34;: \u0026#34;0x0498ac1951b9078a0549c93c3f6088ec7c790032b17580dc3c0c9e900899a48d89eaa27471e3071d2de6a1f48716ecad8b88ee022f4321a7c29b6ffcbee65624ff\u0026#34;, \u0026#34;recipientPublicKey\u0026#34;: null, \u0026#34;ttl\u0026#34;: 10, \u0026#34;timestamp\u0026#34;: 1498577270, \u0026#34;topic\u0026#34;: \u0026#34;0xffaadd11\u0026#34;, \u0026#34;payload\u0026#34;: \u0026#34;0xffffffdddddd1122\u0026#34;, \u0026#34;padding\u0026#34;: \u0026#34;0x35d017b66b124dd4c67623ed0e3f23ba68e3428aa500f77aceb0dbf4b63f69ccfc7ae11e39671d7c94f1ed170193aa3e327158dffdd7abb888b3a3cc48f718773dc0a9dcf1a3680d00fe17ecd4e8d5db31eb9a3c8e6e329d181ecb6ab29eb7a2d9889b49201d9923e6fd99f03807b730780a58924870f541a8a97c87533b1362646e5f573dc48382ef1e70fa19788613c9d2334df3b613f6e024cd7aadc67f681fda6b7a84efdea40cb907371cd3735f9326d02854\u0026#34;, \u0026#34;pow\u0026#34;: 0.6714754098360656, \u0026#34;hash\u0026#34;: \u0026#34;0x17f8066f0540210cf67ef400a8a55bcb32a494a47f91a0d26611c5c1d66f8c57\u0026#34; } } } waku_unsubscribe # Cancels and removes an existing subscription. The node MUST stop sending the client notifications.
Parameters # string - The subscription ID. Response # bool - true or false waku_newMessageFilter # Creates a new message filter within the node. This filter can be used to poll for new messages that match the criteria.
Parameters # The request must contain a message filter as its parameter.
Response # string - The ID of the filter. waku_deleteMessageFilter # Removes a message filter from the node.
Parameters # string - ID of the filter created with waku_newMessageFilter. Response # bool - true on success or an error on failure. waku_getFilterMessages # Retrieves messages that match a filter criteria and were received after the last time this function was called.
Parameters # string - ID of the filter created with waku_newMessageFilter. Response # The response contains an array of messages or an error on failure.
waku_post # The waku_post method creates a waku envelope and propagates it to the network.
Parameters # The parameters is an Object containing the following fields:
symKeyID [string] optional - The ID of the symmetric key used for encryption pubKey [string] optional - The public key for message encryption. sig [string] optional - The ID of the signing key. ttl [number] - The time-to-live in seconds. topic [string] - 4 bytes message topic. payload [string] - The payload to be encrypted. padding [string] optional - The padding, a byte array of arbitrary length. powTime [number] - Maximum time in seconds to be spent on the proof of work. powTarget [number] - Minimal PoW target required for this message. targetPeer [string] optional - The optional peer ID for peer-to-peer messages. Either the symKeyID or the pubKey need to be present. It can not be both.
Response # bool - true on success or an error on failure. Changelog # Version Comment 1.0.0 Initial release. Copyright # Copyright and related rights waived via CC0.
`}),e.add({id:59,href:"/spec/xx/",title:"XX/(WAKU2|LOGOS|CODEX|*)-TEMPLATE",section:"Docs",content:` (Info, remove this section) # This section contains meta info about writing RFCs. This section (including its subsections) MUST be removed.
COSS explains the Vac RFC process.
Tags # The tags metadata SHOULD contain a list of tags if applicable.
Currently identified tags comprise
waku/core-protocol for Waku protocol definitions (e.g. store, relay, light push), waku/application for applications built on top of Waku protocol (e.g. eth-dm, toy-chat), Abstract # Background / Rationale / Motivation # This section serves as an introduction providing background information and a motivation/rationale for why the specified protocol is useful.
Theory / Semantics # A standard track RFC in stable status MUST feature this section. A standard track RFC in raw or draft status SHOULD feature this section. This section SHOULD explain in detail how the proposed protocol works. It may touch on the wire format where necessary for the explanation. This section MAY also specify endpoint behaviour when receiving specific messages, e.g. the behaviour of certain caches etc.
Wire Format Specification / Syntax # A standard track RFC in stable status MUST feature this section. A standard track RFC in raw or draft status SHOULD feature this section. This section SHOULD not contain explanations of semantics and focus on concisely defining the wire format. Implementations MUST adhere to these exact formats to interoperate with other implementations. It is fine, if parts of the previous section that touch on the wire format are repeated. The purpose of this section is having a concise definition of what an implementation sends and accepts. Parts that are not specified here are considered implementation details. Implementors are free to decide on how to implement these details. An optional implementation suggestions section may provide suggestions on how to approach implementation details, and, if available, point to existing implementations for reference.
Implementation Suggestions (optional) # (Further Optional Sections) # Security/Privacy Considerations # A standard track RFC in stable status MUST feature this section. A standard track RFC in raw or draft status SHOULD feature this section. Informational RFCs (in any state) may feature this section. If there are none, this section MUST explicitly state that fact. This section MAY contain additional relevant information, e.g. an explanation as to why there are no security consideration for the respective document.
Copyright # Copyright and related rights waived via CC0.
References # References MAY be subdivided into normative and informative.
normative # A list of references that MUST be read to fully understand and/or implement this protocol. See RFC3967 Section 1.1.
informative # A list of additional references.
`})})()