logos-messaging/specs
1
0
Fork 0
mirror of https://github.com/logos-messaging/specs.git synced 2026-05-05 20:49:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into waku-keystore

Browse Source
This commit is contained in:
Jimmy Debe 2024-07-01 14:01:01 -04:00 committed by GitHub
commit 3cc6b63436
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
17 changed files with 953 additions and 999 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
.github/workflows/spelling.yml vendored
View File

@ -1,4 +1,3 @@
# This is workflow for spell checking using PySpelling lib (https://pypi.org/project/pyspelling/)
name: Spellcheck name: Spellcheck
on: on:
@ -11,12 +10,16 @@ on:
workflow_dispatch: workflow_dispatch:
jobs: jobs:
build: spellcheck:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
- uses: igsekor/pyspelling-any@v1.0.4
name: Spellcheck - name: Install aspell
env: run: sudo apt-get install -y aspell aspell-en
MATRIX: ${{ toJson(matrix) }}
run: pyspelling --matrix "$MATRIX" - name: Run spellcheck
uses: igsekor/pyspelling-any@v1.0.4
with:
args: "-c ./spellcheck.yaml"

100
informational/adversarial-models.md
View File

@ -2,7 +2,7 @@
title: ADVERSARIAL-MODELS title: ADVERSARIAL-MODELS
name: Waku v2 Adversarial Models and Attack-based Threat List name: Waku v2 Adversarial Models and Attack-based Threat List
category: Informational category: Informational
tags: tags: []
editor: Daniel Kaiser <danielkaiser@status.im> editor: Daniel Kaiser <danielkaiser@status.im>
contributors: contributors:
--- ---
@ -17,9 +17,9 @@ Future versions of this document will serve as a comprehensive list of adversari
The main purpose of this document is being a linkable resource for specifications that address protection as well as mitigation mechanisms within the listed models. The main purpose of this document is being a linkable resource for specifications that address protection as well as mitigation mechanisms within the listed models.
Discussing and introducing countermeasures to specific attacks in specific models is out of scope for this document. Discussing and introducing countermeasures to specific attacks in specific models is out of scope for this document.
Analyses and further information about Waku's properties within these models may be found in our *Waku v2 Anonymity Analysis* series of research log posts: Analyses and further information about Waku's properties within these models may be found in our _Waku v2 Anonymity Analysis_ series of research log posts:
* [Part I: Definitions and Waku Relay](https://vac.dev/wakuv2-relay-anon) - [Part I: Definitions and Waku Relay](https://vac.dev/wakuv2-relay-anon)
Note: This document adds to the adversarial models and threat list discussed in our [research log post](https://vac.dev/wakuv2-relay-anon). Note: This document adds to the adversarial models and threat list discussed in our [research log post](https://vac.dev/wakuv2-relay-anon).
It does not cover analysis of Waku, as the research log post does. It does not cover analysis of Waku, as the research log post does.
@ -32,11 +32,11 @@ The concepts of security, privacy, and anonymity are linked and have quite a bit
### Security ### Security
Of the three, [Security](https://en.wikipedia.org/wiki/Information_security) has the clearest agreed upon definition, Of the three, [Security](https://en.wikipedia.org/wiki/Information_security) has the clearest agreed upon definition,
at least regarding its key concepts: *confidentiality*, *integrity*, and *availability*. at least regarding its key concepts: _confidentiality_, _integrity_, and _availability_.
* confidentiality: data is not disclosed to unauthorized entities. - confidentiality: data is not disclosed to unauthorized entities.
* integrity: data is not modified by unauthorized entities. - integrity: data is not modified by unauthorized entities.
* availability: data is available, i.e. accessible by authorized entities. - availability: data is available, i.e. accessible by authorized entities.
While these are the key concepts, the definition of information security has been extended over time including further concepts, While these are the key concepts, the definition of information security has been extended over time including further concepts,
e.g. [authentication](https://en.wikipedia.org/wiki/Authentication) and [non-repudiation](https://en.wikipedia.org/wiki/Non-repudiation). e.g. [authentication](https://en.wikipedia.org/wiki/Authentication) and [non-repudiation](https://en.wikipedia.org/wiki/Non-repudiation).
@ -45,8 +45,8 @@ e.g. [authentication](https://en.wikipedia.org/wiki/Authentication) and [non-rep
Privacy allows users to choose which data and information Privacy allows users to choose which data and information
* they want to share - they want to share
* and with whom they want to share it. - and with whom they want to share it.
This includes data and information that is associated with and/or generated by users. This includes data and information that is associated with and/or generated by users.
Protected data also comprises metadata that might be generated without users being aware of it. Protected data also comprises metadata that might be generated without users being aware of it.
@ -66,28 +66,28 @@ Privacy and anonymity are closely linked.
Both the identity of a user and data that allows inferring a user's identity should be part of the privacy policy. Both the identity of a user and data that allows inferring a user's identity should be part of the privacy policy.
For the purpose of analysis, we want to have a clearer separation between these concepts. For the purpose of analysis, we want to have a clearer separation between these concepts.
We define anonymity as *unlinkablity of users' identities and their shared data and/or actions*. We define anonymity as _unlinkablity of users' identities and their shared data and/or actions_.
We subdivide anonymity into *receiver anonymity* and *sender anonymity*. We subdivide anonymity into _receiver anonymity_ and _sender anonymity_.
#### Receiver Anonymity #### Receiver Anonymity
We define receiver anonymity as *unlinkability of users' identities and the data they receive and/or related actions*. We define receiver anonymity as _unlinkability of users' identities and the data they receive and/or related actions_.
Because each [Waku message](https://rfc.vac.dev/spec/14/) is associated with a content topic, and each receiver is interested in messages with specific content topics, Because each [Waku message](https://rfc.vac.dev/spec/14/) is associated with a content topic, and each receiver is interested in messages with specific content topics,
receiver anonymity in the context of Waku corresponds to *subscriber-topic unlinkability*. receiver anonymity in the context of Waku corresponds to _subscriber-topic unlinkability_.
An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic. An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.
#### Sender Anonymity #### Sender Anonymity
We define sender anonymity as *unlinkability of users' identities and the data they send and/or related actions*. We define sender anonymity as _unlinkability of users' identities and the data they send and/or related actions_.
Because the data in the context of Waku is Waku messages, sender anonymity corresponds to *sender-message unlinkability*. Because the data in the context of Waku is Waku messages, sender anonymity corresponds to _sender-message unlinkability_.
#### Anonymity Trilemma #### Anonymity Trilemma
[The Anonymity trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html) states that only two out of *strong anonymity*, *low bandwidth*, and *low latency* can be guaranteed in the *global attacker* model. [The Anonymity trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html) states that only two out of _strong anonymity_, _low bandwidth_, and _low latency_ can be guaranteed in the _global attacker_ model.
Waku's goal, being a modular set of protocols, is to offer any combination of two out of these three properties, as well as blends. Waku's goal, being a modular set of protocols, is to offer any combination of two out of these three properties, as well as blends.
A fourth factor that influences [the anonymity trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html) is *frequency and patterns* of messages. A fourth factor that influences [the anonymity trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html) is _frequency and patterns_ of messages.
The more messages there are, and the more randomly distributed they are, the better the anonymity protection offered by a given anonymous communication protocol. The more messages there are, and the more randomly distributed they are, the better the anonymity protection offered by a given anonymous communication protocol.
So, incentivising users to use the protocol, for instance by lowering entry barriers, helps protecting the anonymity of all users. So, incentivising users to use the protocol, for instance by lowering entry barriers, helps protecting the anonymity of all users.
The frequency/patterns factor is also related to [k-anonymity](https://en.wikipedia.org/wiki/K-anonymity). The frequency/patterns factor is also related to [k-anonymity](https://en.wikipedia.org/wiki/K-anonymity).
@ -147,7 +147,7 @@ Nodes controlled by the attacker can efficiently communicate out-of-band to coor
### External ### External
An external attacker can only see encrypted traffic. An external attacker can only see encrypted traffic.
Waku protocols are protected by a secure channel set up with [Noise](../standards/core/noise.md). Waku protocols are protected by a secure channel set up with [Noise](../standards/application/noise.md).
#### Local #### Local
@ -162,7 +162,7 @@ An active AS attacker can drop, delay, inject, and alter traffic on arbitrary li
In practice, a malicious ISP would be considered as an AS attacker. In practice, a malicious ISP would be considered as an AS attacker.
A malicious ISP could also easily setup a set of nodes at specific points in the network, A malicious ISP could also easily setup a set of nodes at specific points in the network,
gaining internal attack power similar to a strong *multi node* or even *scaling multi node* attacker. gaining internal attack power similar to a strong _multi node_ or even _scaling multi node_ attacker.
#### Global (On-Net) #### Global (On-Net)
@ -171,12 +171,12 @@ A passive global attacker can listen to traffic on all links,
while the active global attacker basically carries the traffic: it can freely drop, delay, inject, and alter traffic at all positions in the network. while the active global attacker basically carries the traffic: it can freely drop, delay, inject, and alter traffic at all positions in the network.
This basically corresponds to the [Dolev-Yao model](https://en.wikipedia.org/wiki/Dolev%E2%80%93Yao_model). This basically corresponds to the [Dolev-Yao model](https://en.wikipedia.org/wiki/Dolev%E2%80%93Yao_model).
An entity with this power would, in practice, also have the power of the internal *scaling multi node* attacker. An entity with this power would, in practice, also have the power of the internal _scaling multi node_ attacker.
## Attack-based Threats ## Attack-based Threats
The following lists various attacks against [Waku v2](https://rfc.vac.dev/spec/10/) protocols. The following lists various attacks against [Waku v2](https://rfc.vac.dev/spec/10/) protocols.
If not specifically mentioned, the attacks refer to [Waku relay](/spec/11) and the underlying [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md). If not specifically mentioned, the attacks refer to [Waku relay](https://rfc.vac.dev/spec/11/) and the underlying [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
We also list the weakest attacker model in which the attack can be successfully performed against. We also list the weakest attacker model in which the attack can be successfully performed against.
An attack is considered more powerful if it can be successfully performed in a weaker attacker model. An attack is considered more powerful if it can be successfully performed in a weaker attacker model.
@ -194,7 +194,7 @@ In libp2p gossipsub, and by extension Waku v2 relay, nodes can simply send a gra
If the victim node still has open slots, the attacker gets the desired position. If the victim node still has open slots, the attacker gets the desired position.
This only requires the attacker to know the gossipsub multiaddress of the victim node. This only requires the attacker to know the gossipsub multiaddress of the victim node.
A *scaling multi node* attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned, A _scaling multi node_ attacker can leverage DHT based discovery systems to boost the probability of malicious nodes being returned,
which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes. which in turn significantly increases the probability of attacker nodes ending up in the peer lists of victim nodes.
### Sender Deanonymization ### Sender Deanonymization
@ -203,7 +203,7 @@ This section lists attacks that aim at deanonymizing a message sender.
We assume that protocol messages are transmitted within a secure channel set up using the [Noise Protocol Framework](https://noiseprotocol.org/). We assume that protocol messages are transmitted within a secure channel set up using the [Noise Protocol Framework](https://noiseprotocol.org/).
For [Waku Relay](https://rfc.vac.dev/spec/11/) this means we only consider messages with version field `2`, For [Waku Relay](https://rfc.vac.dev/spec/11/) this means we only consider messages with version field `2`,
which indicates that the payload has to be encoded using [Noise](../standards/core/noise.md). which indicates that the payload has to be encoded using [Noise](../standards/application/noise.md).
Note: The currently listed attacks are against libp2p in general. Note: The currently listed attacks are against libp2p in general.
The [data field of Waku v2 relay](https://rfc.vac.dev/spec/11/#message-fields) must be a [Waku v2 message](https://rfc.vac.dev/spec/14/). The [data field of Waku v2 relay](https://rfc.vac.dev/spec/11/#message-fields) must be a [Waku v2 message](https://rfc.vac.dev/spec/14/).
@ -227,7 +227,7 @@ Waku relay protects against this attack by employing secure channels setup using
#### Neighbourhood Surveillance #### Neighbourhood Surveillance
This attack can be performed by a single node attacker that is connected to all peers of the victim node $v$ with respect to a specific topic mesh. This attack can be performed by a single node attacker that is connected to all peers of the victim node $v$ with respect to a specific topic mesh.
The attacker also has to be connected to $v$. The attacker also has to be connected to $v$.
In this position, the attacker will receive messages $m_v$ sent by $v$ both on the direct path from $v$, and on indirect paths relayed by peers of $v$. In this position, the attacker will receive messages $m_v$ sent by $v$ both on the direct path from $v$, and on indirect paths relayed by peers of $v$.
It will also receive messages $m_x$ that are not sent by $v$. These messages $m_x$ are relayed by both $v$ and the peers of $v$. It will also receive messages $m_x$ that are not sent by $v$. These messages $m_x$ are relayed by both $v$ and the peers of $v$.
@ -243,7 +243,7 @@ This attack cannot (reliably) distinguish messages $m_v$ sent by $v$ from messag
Still, there are hop-count variations that can be leveraged. Still, there are hop-count variations that can be leveraged.
Messages $m_v$ always have a hop-count of 1 on the path from $v$ to the attacker, while all other paths are longer. Messages $m_v$ always have a hop-count of 1 on the path from $v$ to the attacker, while all other paths are longer.
Messages $m_y$ might have the same hop-count on the path from $v$ as well as on other paths. Messages $m_y$ might have the same hop-count on the path from $v$ as well as on other paths.
Further techniques that are part of the *mass deanonymization* category, such as [bayesian analysis](#bayesian-analysis), can be used here as well. Further techniques that are part of the _mass deanonymization_ category, such as [bayesian analysis](#bayesian-analysis), can be used here as well.
#### Controlled Neighbourhood #### Controlled Neighbourhood
@ -259,8 +259,8 @@ Combined with just a few nodes controlled by the attacker, the actual message as
### Mass Deanonymization ### Mass Deanonymization
While attacks in the *sender deanonymization* category target a set of either specific or arbitrary users, While attacks in the _sender deanonymization_ category target a set of either specific or arbitrary users,
attacks in the *mass deanonymization* category aim at deanonymizing (parts of) the whole network. attacks in the _mass deanonymization_ category aim at deanonymizing (parts of) the whole network.
Mass deanonymization attacks do not necessarily link messages to senders. Mass deanonymization attacks do not necessarily link messages to senders.
They might only reduce the anonymity set in which senders hide, They might only reduce the anonymity set in which senders hide,
or infer information about the network topology. or infer information about the network topology.
@ -269,7 +269,7 @@ or infer information about the network topology.
Graph learning attacks are a prerequisite for some mass deanonymization attacks, Graph learning attacks are a prerequisite for some mass deanonymization attacks,
in which the attacker learns the overlay network topology. in which the attacker learns the overlay network topology.
Graph learning attacks require a *scaling multinode* attacker Graph learning attacks require a _scaling multinode_ attacker
For gossipsub this means an attacker learns the topic mesh for specific pubsub topics. For gossipsub this means an attacker learns the topic mesh for specific pubsub topics.
[Dandelion++](https://arxiv.org/abs/1805.11060) describes ways to perform this attack. [Dandelion++](https://arxiv.org/abs/1805.11060) describes ways to perform this attack.
@ -278,8 +278,8 @@ For gossipsub this means an attacker learns the topic mesh for specific pubsub t
Bayesian analysis allows attackers to assign each node in the network a likelihood of having sent (originated) a specific message. Bayesian analysis allows attackers to assign each node in the network a likelihood of having sent (originated) a specific message.
Bayesian analysis for mass deanonymization is detailed in [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860). Bayesian analysis for mass deanonymization is detailed in [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860).
It requires a *scaling node* attacker as well as knowledge of the network topology, It requires a _scaling node_ attacker as well as knowledge of the network topology,
which can be learned via *graph learning* attacks. which can be learned via _graph learning_ attacks.
### Denial of Service (DoS) ### Denial of Service (DoS)
@ -295,8 +295,8 @@ Waku employs [RLN Relay](https://rfc.vac.dev/spec/17/) as the main countermeasur
In a black hole attack, the attacker does not relay messages it is supposed to relay. In a black hole attack, the attacker does not relay messages it is supposed to relay.
Analogous to a black hole, attacker nodes do not allow messages to leave once they entered. Analogous to a black hole, attacker nodes do not allow messages to leave once they entered.
While *single node* and smaller *multi node* attackers can have a negative effect on availability, the impact is not significant. While _single node_ and smaller _multi node_ attackers can have a negative effect on availability, the impact is not significant.
A *scaling multi node* attacker, however, can significantly disrupt the network with such an attack. A _scaling multi node_ attacker, however, can significantly disrupt the network with such an attack.
The effects of this attack are especially severe in conjunction with deanonymization mitigation techniques that reduce the out-degree of the overlay, The effects of this attack are especially severe in conjunction with deanonymization mitigation techniques that reduce the out-degree of the overlay,
such as [Waku Dandelion](../standards/application/dandelion.md). such as [Waku Dandelion](../standards/application/dandelion.md).
@ -308,7 +308,7 @@ A local attacker can filter and drop all Waku traffic within its controlled netw
An AS attacker can filter and drop all Waku traffic within its authority, while a global attacker can censor the whole network. An AS attacker can filter and drop all Waku traffic within its authority, while a global attacker can censor the whole network.
A countermeasure are censorship resistance techniques like [Pluggable Transports](https://www.pluggabletransports.info/about/). A countermeasure are censorship resistance techniques like [Pluggable Transports](https://www.pluggabletransports.info/about/).
An entity trying to censor Waku can employ both the *black hole* attack and *traffic filtering*; An entity trying to censor Waku can employ both the _black hole_ attack and _traffic filtering_;
the former is internal while the latter is external. the former is internal while the latter is external.
## Copyright ## Copyright
@ -317,20 +317,20 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
* [10/WAKU2](https://rfc.vac.dev/spec/10/) - [10/WAKU2](https://rfc.vac.dev/spec/10/)
* [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/) - [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/)
* [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) - [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
* [Security](https://en.wikipedia.org/wiki/Information_security) - [Security](https://en.wikipedia.org/wiki/Information_security)
* [Authentication](https://en.wikipedia.org/wiki/Authentication) - [Authentication](https://en.wikipedia.org/wiki/Authentication)
* [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html) - [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html)
* [Waku v2 message](https://rfc.vac.dev/spec/14/) - [Waku v2 message](https://rfc.vac.dev/spec/14/)
* [Pluggable Transports](https://www.pluggabletransports.info/about/) - [Pluggable Transports](https://www.pluggabletransports.info/about/)
* [Sybil attack](https://en.wikipedia.org/wiki/Sybil_attack) - [Sybil attack](https://en.wikipedia.org/wiki/Sybil_attack)
* [Dolev-Yao model](https://en.wikipedia.org/wiki/Dolev%E2%80%93Yao_model) - [Dolev-Yao model](https://en.wikipedia.org/wiki/Dolev%E2%80%93Yao_model)
* [Noise Protocol Framework](https://noiseprotocol.org/) - [Noise Protocol Framework](https://noiseprotocol.org/)
* [Noise](../standards/core/noise.md) - [Noise](../standards/application/noise.md)
* [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/) - [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/)
* [18/WAKU2-SWAP](https://rfc.vac.dev/spec/18/) - [18/WAKU2-SWAP](https://rfc.vac.dev/spec/18/)
* [Dandelion++](https://arxiv.org/abs/1805.11060) - [Dandelion++](https://arxiv.org/abs/1805.11060)
* [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860) - [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860)
* [Waku Dandelion](../standards/application/dandelion.md)) - [Waku Dandelion](../standards/application/dandelion.md))

52
informational/relay-static-shard-alloc.md
View File

@ -3,7 +3,7 @@ title: RELAY-STATIC-SHARD-ALLOC
name: Waku v2 Relay Static Shard Allocation name: Waku v2 Relay Static Shard Allocation
status: raw status: raw
category: Informational category: Informational
tags: waku/informational tags: [waku/informational]
editor: Daniel Kaiser <danielkaiser@status.im> editor: Daniel Kaiser <danielkaiser@status.im>
contributors: contributors:
--- ---
@ -19,32 +19,31 @@ this document lists static shard index assignments (see [WAKU2-RELAY-SHARDING](.
## Assingment Process ## Assingment Process
> *Note*: Future versions of this document will specify the assignment process. > _Note_: Future versions of this document will specify the assignment process.
### List of Cluster Ids ### List of Cluster Ids
| index | Protocol/App | Description | | index | Protocol/App | Description |
| --- | --- | --- | | ----- | ------------ | --------------------------------------------------------------- |
| 0 | global | global use | | 0 | global | global use |
| 1 | reserved | [The Waku Network](https://rfc.vac.dev/spec/64/#network-shards) | | 1 | reserved | [The Waku Network](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md) |
| 2 | reserved | | | 2 | reserved | |
| 3 | reserved | | | 3 | reserved | |
| 4 | reserved | | | 4 | reserved | |
| 5 | reserved | | | 5 | reserved | |
| 6 | reserved | | | 6 | reserved | |
| 7 | reserved | | | 7 | reserved | |
| 8 | reserved | | | 8 | reserved | |
| 9 | reserved | | | 9 | reserved | |
| 10 | reserved | | | 10 | reserved | |
| 11 | reserved | | | 11 | reserved | |
| 12 | reserved | | | 12 | reserved | |
| 13 | reserved | | | 13 | reserved | |
| 14 | reserved | | | 14 | reserved | |
| 15 | reserved | | | 15 | reserved | |
| 16 | Status | Status main net | | 16 | Status | Status main net |
| 17 | Status | | | 17 | Status | |
| 18 | Status | | | 18 | Status | |
## Copyright ## Copyright
@ -52,5 +51,6 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
* [WAKU2-RELAY-SHARDING](../standards/core/relay-sharding.md) - [WAKU2-RELAY-SHARDING](../standards/core/relay-sharding.md)
* [IANA port allocation](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml) - [IANA port allocation](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml)
- [The Waku Network](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md)

36
spellcheck.yaml
View File

@ -2,3 +2,39 @@ matrix:
- name: markdown - name: markdown
sources: sources:
- '**/*.md' - '**/*.md'
default:
dictionary:
- en_US
- en_GB
encoding: utf-8
pipeline:
- pyspelling.filters.markdown:
extensions:
- .md
- pyspelling.filters.text
sources:
- '**/*.md'
suggest: true
whitelist:
- ALLOC
- IANA
- SHARDING
- WAKU
- Waku
- danielkaiser
- creativecommons
- github
- GITHUB
- https
- iana
- md
- rfc
- RFC
- www
- DHT
- DoS
- GossipSub
- gossipsub
- libp2p
- pubsub
- subnets

52
standards/application/dandelion.md
View File

@ -2,7 +2,7 @@
title: DANDELION title: DANDELION
name: Waku v2 Dandelion name: Waku v2 Dandelion
category: Standards Track category: Standards Track
tags: waku/anonymity tags: [waku/anonymity]
editor: Daniel Kaiser <danielkaiser@status.im> editor: Daniel Kaiser <danielkaiser@status.im>
contributors: contributors:
--- ---
@ -71,12 +71,12 @@ Adding random delay in the fluff phase further reduces symmetry in dissemination
introduces more uncertainty for the attacker. introduces more uncertainty for the attacker.
Specifying fluff phase augmentations is out of scope for this document. Specifying fluff phase augmentations is out of scope for this document.
*Note: _Note:
We plan to add a separate specification for fluff phase augmentations. We plan to add a separate specification for fluff phase augmentations.
We envision stem and fluff phase as abstract concepts. We envision stem and fluff phase as abstract concepts.
The Dandelion stem and fluff phases instantiate these concepts. The Dandelion stem and fluff phases instantiate these concepts.
Future stem specifications might comprise: none (standard relay), Dandelion stem, Tor, and mix-net. Future stem specifications might comprise: none (standard relay), Dandelion stem, Tor, and mix-net.
As for future fluff specifications: none (standard relay), diffusion (random delays), and mix-net.* As for future fluff specifications: none (standard relay), diffusion (random delays), and mix-net._
Messages relayed by nodes supporting 44/WAKU2-DANDELION are either in stem phase or in fluff phase. Messages relayed by nodes supporting 44/WAKU2-DANDELION are either in stem phase or in fluff phase.
We refer to the former as a stem message and to the latter as a fluff message. We refer to the former as a stem message and to the latter as a fluff message.
@ -95,7 +95,7 @@ once they arrive at a node in fluff state for the first time.
44/WAKU2-DANDELION uses [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) as the protocol for relaying stem messages. 44/WAKU2-DANDELION uses [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) as the protocol for relaying stem messages.
There are no negative effects on gossipsub peer scoring, There are no negative effects on gossipsub peer scoring,
because Dandelion nodes in *stem state* still normally relay Waku Relay (gossipsub) messages. because Dandelion nodes in _stem state_ still normally relay Waku Relay (gossipsub) messages.
## Specification ## Specification
@ -136,7 +136,7 @@ Received fluff messages MUST be relayed as specified in the fluff state section.
The stem protocol ([19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/)) is independent of the fluff protocol ([Waku Relay](https://rfc.vac.dev/spec/11/)). The stem protocol ([19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/)) is independent of the fluff protocol ([Waku Relay](https://rfc.vac.dev/spec/11/)).
While in stem state, nodes MUST NOT gossip about stem messages, While in stem state, nodes MUST NOT gossip about stem messages,
and MUST NOT send control messages related to stem messages. and MUST NOT send control messages related to stem messages.
(An existing gossipsub implementation does *not* have to be adjusted to not send gossip about stem messages, (An existing gossipsub implementation does _not_ have to be adjusted to not send gossip about stem messages,
because these messages are only handed to gossipsub once they enter fluff phase.) because these messages are only handed to gossipsub once they enter fluff phase.)
#### Fail Safe #### Fail Safe
@ -144,8 +144,8 @@ because these messages are only handed to gossipsub once they enter fluff phase.
Nodes $v$ in stem state SHOULD store messages attached with a random timer between $t_1 = 5 * 100ms$ and $t_2 = 2 * t_1$. Nodes $v$ in stem state SHOULD store messages attached with a random timer between $t_1 = 5 * 100ms$ and $t_2 = 2 * t_1$.
This time interval is chosen because This time interval is chosen because
* we assume $100\,ms$ as an average per hop delay, and - we assume $100\,ms$ as an average per hop delay, and
* using $q=0.2$ will lead to an expected number of 5 stem hops per message. - using $q=0.2$ will lead to an expected number of 5 stem hops per message.
If $v$ does not receive a given message via Waku Relay (fluff) before the respective timer runs out, If $v$ does not receive a given message via Waku Relay (fluff) before the respective timer runs out,
$v$ will disseminate the message via Waku Relay. $v$ will disseminate the message via Waku Relay.
@ -188,7 +188,7 @@ in which the attacker controls a certain percentage of nodes in the network.
44/WAKU2-DANDELION provides significant mitigation against mass deanonymization 44/WAKU2-DANDELION provides significant mitigation against mass deanonymization
even if the attacker knows the network topology, i.e. the anonymity graph and the relay mesh graph. even if the attacker knows the network topology, i.e. the anonymity graph and the relay mesh graph.
Mitigation in stronger models, including the *active scaling multi-node* model, is weak. Mitigation in stronger models, including the _active scaling multi-node_ model, is weak.
We will elaborate on this in future versions of this document. We will elaborate on this in future versions of this document.
44/WAKU2-DANDELION does not protect against targeted deanonymization attacks. 44/WAKU2-DANDELION does not protect against targeted deanonymization attacks.
@ -211,10 +211,10 @@ This is still work in progress and will be elaborated on in future versions of t
Generally, there are several design choices to be made for the stem phase of a Dandelion-based specification: Generally, there are several design choices to be made for the stem phase of a Dandelion-based specification:
1) the probability of continuing the stem phase, which determines the expected stem lengh, 1. the probability of continuing the stem phase, which determines the expected stem lengh,
2) the out degree in the stem phase, which set to 1 in this document (also in the Dandelion papers), 2. the out degree in the stem phase, which set to 1 in this document (also in the Dandelion papers),
3) the rate of re-selecting stem relays among all gossipsub mesh peers (for a given pubsub topic), and 3. the rate of re-selecting stem relays among all gossipsub mesh peers (for a given pubsub topic), and
4) the mapping of incoming connections to outgoing connections. 4. the mapping of incoming connections to outgoing connections.
#### Bound Stem Length #### Bound Stem Length
@ -272,9 +272,9 @@ While this improves anonymity, as discussed above, it also introduces additional
In future versions of this specification we might In future versions of this specification we might
* add a flag to [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) indicating a message should be routed over a Dandelion stem (opt-in), or - add a flag to [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) indicating a message should be routed over a Dandelion stem (opt-in), or
* add a flag to [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) indicating a message should *not* be routed over a Dandelion stem (opt-out), or - add a flag to [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) indicating a message should _not_ be routed over a Dandelion stem (opt-out), or
* introducing a fork of [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) exclusively used for Dandelion stem. - introducing a fork of [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) exclusively used for Dandelion stem.
In the current version, we decided against these options in favour of a simpler protocol and an increased anonymity set. In the current version, we decided against these options in favour of a simpler protocol and an increased anonymity set.
@ -284,14 +284,14 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
* [Dandelion](https://arxiv.org/abs/1701.04439) - [Dandelion](https://arxiv.org/abs/1701.04439)
* [Dandelion++](https://arxiv.org/abs/1805.11060) - [Dandelion++](https://arxiv.org/abs/1805.11060)
* [multi-node (botnet) attacker model](../../informational/adversarial-models.md/#multi-node) - [multi-node (botnet) attacker model](../../informational/adversarial-models.md/#multi-node)
* [Waku Relay](https://rfc.vac.dev/spec/11/) - [Waku Relay](https://rfc.vac.dev/spec/11/)
* [Waku v2](https://rfc.vac.dev/spec/10/) - [Waku v2](https://rfc.vac.dev/spec/10/)
* [d-regular graph](https://en.wikipedia.org/wiki/Regular_graph) - [d-regular graph](https://en.wikipedia.org/wiki/Regular_graph)
* [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html) - [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html)
* [Waku Privacy and Anonymity Analysis](https://vac.dev/wakuv2-relay-anon). - [Waku Privacy and Anonymity Analysis](https://vac.dev/wakuv2-relay-anon).
* [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860.pdf) - [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860.pdf)
* [Adversarial Models](https://rfc.vac.dev/spec/45/) - [Adversarial Models](https://rfc.vac.dev/spec/45/)
* [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) - [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/)

295
standards/application/device-pairing.md
View File

@ -2,42 +2,42 @@
title: WAKU2-DEVICE-PAIRING title: WAKU2-DEVICE-PAIRING
name: Device pairing and secure transfers with Noise name: Device pairing and secure transfers with Noise
category: Standards Track category: Standards Track
tags: waku/core-protocol tags: [waku/core-protocol]
editor: Giuseppe <giuseppe@status.im> editor: Giuseppe <giuseppe@status.im>
contributors: contributors:
--- ---
## Abstract ## Abstract
In this document we describe a compound protocol In this document we describe a compound protocol
for enabling two devices to mutually authenticate for enabling two devices to mutually authenticate
and securely exchange (arbitrary) information over the Waku network. and securely exchange (arbitrary) information over the Waku network.
## Background / Rationale / Motivation ## Background / Rationale / Motivation
In order to implement multi-device communications using one of the Noise session management mechanisms proposed in [WAKU2-NOISE-SESSIONS](./noise-sessions/noise-sessions.md), In order to implement multi-device communications using one of the Noise session management mechanisms proposed in [WAKU2-NOISE-SESSIONS](./noise-sessions.md),
we require a protocol to securely exchange (cryptographic) information between 2 or more devices possessed by a user. we require a protocol to securely exchange (cryptographic) information between 2 or more devices possessed by a user.
Since, in this scenario, the devices would be close to each other, Since, in this scenario, the devices would be close to each other,
authentication can be initialized by exchanging a QR code out-of-band authentication can be initialized by exchanging a QR code out-of-band
and then securely completed over the Waku network. and then securely completed over the Waku network.
The protocol we propose consists of two main subprotocols or *phases*: The protocol we propose consists of two main subprotocols or _phases_:
- [Device Pairing](#Device-Pairing): two phisically close devices initialize the *pairing* by exchanging a QR code out-of-band. The devices then exchange and authenticate their respective long-term device ID static key by exchanging handshake messages over the Waku network; - [Device Pairing](#Device-Pairing): two phisically close devices initialize the _pairing_ by exchanging a QR code out-of-band. The devices then exchange and authenticate their respective long-term device ID static key by exchanging handshake messages over the Waku network;
- [Secure Transfer](#Secure-Transfer): the devices securely exchange information in encrypted form using key material obtained during a successful pairing phase. The communication will happen over the Waku network, hence the devices do not need to be phisically close in this phase. - [Secure Transfer](#Secure-Transfer): the devices securely exchange information in encrypted form using key material obtained during a successful pairing phase. The communication will happen over the Waku network, hence the devices do not need to be phisically close in this phase.
## Theory / Semantics ## Theory / Semantics
### Device Pairing ### Device Pairing
In the pairing phase, device `B` requests to be paired to a device `A`. In the pairing phase, device `B` requests to be paired to a device `A`.
Once the two devices are paired, the devices will be mutually authenticated Once the two devices are paired, the devices will be mutually authenticated
and will share a Noise session within which they can securely exchange information. and will share a Noise session within which they can securely exchange information.
The request is made by exposing a QR code that, by default, has to be scanned by device `A`. The request is made by exposing a QR code that, by default, has to be scanned by device `A`.
If device `A` doesn't have a camera while device `B` does, If device `A` doesn't have a camera while device `B` does,
[it is possible](#Rationale) to execute a slightly different pairing (with same security guarantees), [it is possible](#Rationale) to execute a slightly different pairing (with same security guarantees),
where `A` is exposing a QR code instead. where `A` is exposing a QR code instead.
This protocol is designed in order to achieve two main security objectives: This protocol is designed in order to achieve two main security objectives:
@ -53,8 +53,8 @@ This protocol is designed in order to achieve two main security objectives:
#### The `WakuPairing` Noise Handshake #### The `WakuPairing` Noise Handshake
The devices execute a custom handshake derived from `XX`, The devices execute a custom handshake derived from `XX`,
where they mutually exchange and authenticate their respective device static key where they mutually exchange and authenticate their respective device static key
by exchanging messages over the content topic with the following [format](https://rfc.vac.dev/spec/23/#content-topic-format) by exchanging messages over the content topic with the following [format](https://rfc.vac.dev/spec/23/#content-topic-format)
``` ```
@ -77,83 +77,89 @@ d. -> sA, sAeB, sAsB {s}
#### Protocol Flow #### Protocol Flow
1. The device `B` exposes through a QR code a [base64 (url safe)](https://datatracker.ietf.org/doc/html/rfc4648#section-5) serialization of: 1. The device `B` exposes through a QR code a [base64 (url safe)](https://datatracker.ietf.org/doc/html/rfc4648#section-5) serialization of:
- An ephemeral public key `eB`;
- The content topic parameters `contentTopicParams = {application-name}, {application-version}, {shard-id}`. - An ephemeral public key `eB`;
- A (randomly generated) 16-bytes long `messageNametag`. - The content topic parameters `contentTopicParams = {application-name}, {application-version}, {shard-id}`.
- A commitment `H(sB||r)` for its static key `sB` where `r` is a random fixed-lenght value. - A (randomly generated) 16-bytes long `messageNametag`.
- A commitment `H(sB||r)` for its static key `sB` where `r` is a random fixed-lenght value.
2. The device `A`: 2. The device `A`:
- scans the QR code;
- obtains `eB`, `contentTopicParams`, `messageNametag`, `Hash(sB||r)`; - scans the QR code;
- checks if `{application-name}` and `{application-version}` from `contentTopicParams` match the local application name and version: if not, aborts the pairing. Sets `contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto`; - obtains `eB`, `contentTopicParams`, `messageNametag`, `Hash(sB||r)`;
- initializes the Noise handshake by passing `contentTopicParams`, `messageNametag` and `Hash(sB||r)` to the handshake prologue; - checks if `{application-name}` and `{application-version}` from `contentTopicParams` match the local application name and version: if not, aborts the pairing. Sets `contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto`;
- executes the pre-handshake message, i.e. processes the key `eB`; - initializes the Noise handshake by passing `contentTopicParams`, `messageNametag` and `Hash(sB||r)` to the handshake prologue;
- executes the first handshake message over `contentTopic`, i.e. - executes the pre-handshake message, i.e. processes the key `eB`;
- processes and sends a Waku message containing an ephemeral key `eA`; - executes the first handshake message over `contentTopic`, i.e.
- performs `DH(eA,eB)` (which computes a symmetric encryption key); - processes and sends a Waku message containing an ephemeral key `eA`;
- attaches as payload to the handshake message the (encrypted) commitment `H(sA||s)` for `A`'s static key `sA`, where `s` is a random fixed-length value; - performs `DH(eA,eB)` (which computes a symmetric encryption key);
- an 8-digits authorization code `authcode` obtained as `HKDF(h) mod 10^8` is displayed on the device, where `h` is the [handshake hash value](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) obtained once the first handshake message is processed. - attaches as payload to the handshake message the (encrypted) commitment `H(sA||s)` for `A`'s static key `sA`, where `s` is a random fixed-length value;
- an 8-digits authorization code `authcode` obtained as `HKDF(h) mod 10^8` is displayed on the device, where `h` is the [handshake hash value](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) obtained once the first handshake message is processed.
3. The device `B`: 3. The device `B`:
- sets `contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto`;
- listens to messages sent to `contentTopic` and locally filters only those with [Waku payload](./noise.md/#abnf) starting with `messageNametag`. If any, continues.
- initializes the Noise handshake by passing `contentTopicParams`, `messageNametag` and `Hash(sB||r)` to the handshake prologue;
- executes the pre-handshake message, i.e. processes its ephemeral key `eB`;
- executes the first handshake message, i.e.
- obtains from the received message a public key `eA`. If `eA` is not a valid public key, the protocol is aborted.
- performs `DH(eA,eB)` (which computes a symmetric encryption key);
- decrypts the commitment `H(sA||s)` for `A`'s static key `sA`.
- an 8 decimal digits authorization code `authcode` obtained as `HKDF(h) mod 10^8` is displayed on the device, where `h`is the [handshake hash value](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) obtained once the first handshake message is processed.
4. Device `A` and `B` wait for the user to confirm with an interaction (button press) - sets `contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto`;
that the authorization code displayed on both devices are the same. - listens to messages sent to `contentTopic` and locally filters only those with [Waku payload](./noise.md/#abnf) starting with `messageNametag`. If any, continues.
If not, the protocol is aborted. - initializes the Noise handshake by passing `contentTopicParams`, `messageNametag` and `Hash(sB||r)` to the handshake prologue;
- executes the pre-handshake message, i.e. processes its ephemeral key `eB`;
- executes the first handshake message, i.e.
- obtains from the received message a public key `eA`. If `eA` is not a valid public key, the protocol is aborted.
- performs `DH(eA,eB)` (which computes a symmetric encryption key);
- decrypts the commitment `H(sA||s)` for `A`'s static key `sA`.
- an 8 decimal digits authorization code `authcode` obtained as `HKDF(h) mod 10^8` is displayed on the device, where `h`is the [handshake hash value](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) obtained once the first handshake message is processed.
4. Device `A` and `B` wait for the user to confirm with an interaction (button press)
that the authorization code displayed on both devices are the same.
If not, the protocol is aborted.
5. The device `B`: 5. The device `B`:
- executes the second handshake message, i.e.
- processes and sends his (encrypted) device static key `sB` over `contentTopic`; - executes the second handshake message, i.e.
- performs `DH(eA,sB)` (which updates the symmetric encryption key); - processes and sends his (encrypted) device static key `sB` over `contentTopic`;
- attaches as payload the (encrypted) commitment randomness `r` used to compute `H(sB||r)`. - performs `DH(eA,sB)` (which updates the symmetric encryption key);
- attaches as payload the (encrypted) commitment randomness `r` used to compute `H(sB||r)`.
6. The device `A`: 6. The device `A`:
- listens to messages sent to `contentTopic` and locally filters only those with Waku payload starting with `messageNametag`. If any, continues.
- decrypts the received message and obtains the public key `sB`. If `sB` is not a valid public key, the protocol is aborted. - listens to messages sent to `contentTopic` and locally filters only those with Waku payload starting with `messageNametag`. If any, continues.
- performs `DH(eA,sB)` (which updates a symmetric encryption key); - decrypts the received message and obtains the public key `sB`. If `sB` is not a valid public key, the protocol is aborted.
- decrypts the payload to obtain the randomness `r`. - performs `DH(eA,sB)` (which updates a symmetric encryption key);
- computes `H(sB||r)` and checks if this value corresponds to the commitment obtained in step 2. If not, the protocol is aborted. - decrypts the payload to obtain the randomness `r`.
- executes the third handshake message, i.e. - computes `H(sB||r)` and checks if this value corresponds to the commitment obtained in step 2. If not, the protocol is aborted.
- processes and sends his (encrypted) device static key `sA` over `contentTopic`; - executes the third handshake message, i.e.
- performs `DH(sA,eB)` (which updates the symmetric encryption key); - processes and sends his (encrypted) device static key `sA` over `contentTopic`;
- performs `DH(sA,sB)` (which updates the symmetric encryption key); - performs `DH(sA,eB)` (which updates the symmetric encryption key);
- attaches as payload the (encrypted) commitment randomness `s` used to compute `H(sA||s)`. - performs `DH(sA,sB)` (which updates the symmetric encryption key);
- calls [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) and obtains two cipher states to encrypt inbound and outbound messages. - attaches as payload the (encrypted) commitment randomness `s` used to compute `H(sA||s)`.
- calls [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) and obtains two cipher states to encrypt inbound and outbound messages.
7. The device `B`: 7. The device `B`:
- listens to messages sent to `contentTopic` and locally filters only those with Waku payload starting with `messageNametag`. If any, continues. - listens to messages sent to `contentTopic` and locally filters only those with Waku payload starting with `messageNametag`. If any, continues.
- obtains from decrypting the received message a public key `sA`. If `sA` is not a valid public key, the protocol is aborted. - obtains from decrypting the received message a public key `sA`. If `sA` is not a valid public key, the protocol is aborted.
- performs `DH(sA,eB)` (which updates a symmetric encryption key); - performs `DH(sA,eB)` (which updates a symmetric encryption key);
- performs `DH(sA,sB)` (which updates a symmetric encryption key); - performs `DH(sA,sB)` (which updates a symmetric encryption key);
- decrypts the payload to obtain the randomness `s`. - decrypts the payload to obtain the randomness `s`.
- Computes `H(sA||s)` and checks if this value corresponds to the commitment obtained in step 3. If not, the protocol is aborted. - Computes `H(sA||s)` and checks if this value corresponds to the commitment obtained in step 3. If not, the protocol is aborted.
- Calls [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) and obtains two cipher states to encrypt inbound and outbound messages. - Calls [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) and obtains two cipher states to encrypt inbound and outbound messages.
#### The `WakuPairing` for Devices without a Camera #### The `WakuPairing` for Devices without a Camera
In the above pairing handshake, the QR is by default exposed by device `B` and not by `A` In the above pairing handshake, the QR is by default exposed by device `B` and not by `A`
because in most use-cases we foresee, the secure transfer phase would consist in because in most use-cases we foresee, the secure transfer phase would consist in
exchanging a single message (e.g., Noise sessions, cryptographic keys, signatures, etc.) from device `A` to `B`. exchanging a single message (e.g., Noise sessions, cryptographic keys, signatures, etc.) from device `A` to `B`.
However, since the user(s) confirm(s) at the end of message `b.` that the authorization code is the same on both devices, However, since the user(s) confirm(s) at the end of message `b.` that the authorization code is the same on both devices,
the role of the handhsake initiator and responder can be safely swapped in message `a.` and `b.`. the role of the handhsake initiator and responder can be safely swapped in message `a.` and `b.`.
Indeed, if the pairing phase successfully completes on both devices, Indeed, if the pairing phase successfully completes on both devices,
the authentication code, the committed static keys and the Noise processing rules will ensure that no Man-in-the-Middle attack took place the authentication code, the committed static keys and the Noise processing rules will ensure that no Man-in-the-Middle attack took place
and that messages can be securely exchanged bi-directionally in the transfer phase. and that messages can be securely exchanged bi-directionally in the transfer phase.
This allows pairing in case device `A` does not have a camera to scan a QR (e.g. a desktop client) while device `B` has. This allows pairing in case device `A` does not have a camera to scan a QR (e.g. a desktop client) while device `B` has.
The resulting handshake would then be: The resulting handshake would then be:
``` ```
WakuPairing2: WakuPairing2:
a. -> eA {H(sB||r), contentTopicParams, messageNametag} a. -> eA {H(sB||r), contentTopicParams, messageNametag}
@ -167,14 +173,14 @@ d. -> sA, sAeB, sAsB {s}
## Secure Transfer ## Secure Transfer
The pairing phase is designed to be application-agnostic The pairing phase is designed to be application-agnostic
and should be flexible enough to mutually authenticate and should be flexible enough to mutually authenticate
and allow exchange of cryptographic key material and allow exchange of cryptographic key material
between two devices over a distributed network of Waku2 nodes. between two devices over a distributed network of Waku2 nodes.
Once the handshake is concluded, Once the handshake is concluded,
(privacy-sensitive) information can be exchanged using the encryption keys agreed upon the pairing phase. (privacy-sensitive) information can be exchanged using the encryption keys agreed upon the pairing phase.
If stronger security guarantees are required, If stronger security guarantees are required,
some [additional tweaks](#Implementation-Suggestions) are possible. some [additional tweaks](#Implementation-Suggestions) are possible.
## Implementation Suggestions ## Implementation Suggestions
@ -182,26 +188,27 @@ some [additional tweaks](#Implementation-Suggestions) are possible.
### Timebox QR exposure ### Timebox QR exposure
We suggest to timebox the exposure of each pairing QR code to few seconds, e.g. 30. We suggest to timebox the exposure of each pairing QR code to few seconds, e.g. 30.
After this time limit, a QR code containing a new ephemeral key, random static key commitment and message nametag (content topic parameters could remain the same) After this time limit, a QR code containing a new ephemeral key, random static key commitment and message nametag (content topic parameters could remain the same)
should replace the previously exposed QR, which can then be discarded. should replace the previously exposed QR, which can then be discarded.
The reason for such suggestion is due to the fact that if an attacker is able to compromise one of the ephemeral keys, The reason for such suggestion is due to the fact that if an attacker is able to compromise one of the ephemeral keys,
he might successfully realize an undetected MitM attack up to the `authcode` confirmation he might successfully realize an undetected MitM attack up to the `authcode` confirmation
(we note that compromising ephemeral keys is outside our and Noise security assumptions). (we note that compromising ephemeral keys is outside our and Noise security assumptions).
The attacker could indeed proceed as follows: The attacker could indeed proceed as follows:
- intercepts the QR; - intercepts the QR;
- blocks/delays the delivery of the pairing message `b.`; - blocks/delays the delivery of the pairing message `b.`;
- compromises `A` or `B` ephemeral key; - compromises `A` or `B` ephemeral key;
- recovers the genuine `authcode` that would have been generated by `A` and `B`; - recovers the genuine `authcode` that would have been generated by `A` and `B`;
- generates ~`10^8` random `t` values until the Noise processing of the message `b'. -> eC, eCeB {H(sC||t)} `, where `eC` and `sC` are the attacker ephemeral and static key, respectively, results in computing the same `authcode` as the one between `A` and `B`; - generates ~`10^8` random `t` values until the Noise processing of the message `b'. -> eC, eCeB {H(sC||t)} `, where `eC` and `sC` are the attacker ephemeral and static key, respectively, results in computing the same `authcode` as the one between `A` and `B`;
- delivers the message `b'. -> eC, eCeB {H(sC||t)}` to `B` (before `A` is able to deliver its message `b.`). - delivers the message `b'. -> eC, eCeB {H(sC||t)}` to `B` (before `A` is able to deliver its message `b.`).
At this point `A` and `B` will observe the same `authcode` (and would then confirm it), At this point `A` and `B` will observe the same `authcode` (and would then confirm it),
but `B` will process the attacker's ephemeral key `eC` instead of `eA`. but `B` will process the attacker's ephemeral key `eC` instead of `eA`.
However, the attacker would not be able to open to device `A` the static key commitment `H(sB||s)` sent by device `B` out-of-band, However, the attacker would not be able to open to device `A` the static key commitment `H(sB||s)` sent by device `B` out-of-band,
and the pairing will abort on `A` side before it reveals its static key. and the pairing will abort on `A` side before it reveals its static key.
Device `B`, instead, will successfully complete the pairing with the attacker. Device `B`, instead, will successfully complete the pairing with the attacker.
Hence, timeboxing the QR exposure, Hence, timeboxing the QR exposure,
@ -209,23 +216,24 @@ also in combination with increasing the number of decimal digits of the `authcod
will strongly limit the probability that an attacker can successfully impersonate device `A` to `B`. will strongly limit the probability that an attacker can successfully impersonate device `A` to `B`.
We stress once more, that such attack requires the compromise of an ephemeral key (outside our security model) We stress once more, that such attack requires the compromise of an ephemeral key (outside our security model)
and that device `A` will in any case detect a mismatch and abort the pairing, and that device `A` will in any case detect a mismatch and abort the pairing,
regardless of the fact that the QR timeboxing mitigation is implemented or not. regardless of the fact that the QR timeboxing mitigation is implemented or not.
### Randomized Rekey ### Randomized Rekey
The Noise Protocol framework supports [`Rekey()`](http://www.noiseprotocol.org/noise.html#rekey) The Noise Protocol framework supports [`Rekey()`](http://www.noiseprotocol.org/noise.html#rekey)
in order to update encryption keys *"so that a compromise of cipherstate keys will not decrypt older* \[exchanged\] *messages"*. in order to update encryption keys _"so that a compromise of cipherstate keys will not decrypt older_ \[exchanged\] _messages"_.
However, if a certain cipherstate key is compromised, However, if a certain cipherstate key is compromised,
it will be possible for the attacker not only to decrypt messages encrypted under that key, it will be possible for the attacker not only to decrypt messages encrypted under that key,
but also all those messages encrypted under any successive new key obtained through a call to `Rekey()`. but also all those messages encrypted under any successive new key obtained through a call to `Rekey()`.
This could be mitigated by attaching an ephemeral key to messages sent after a [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) This could be mitigated by attaching an ephemeral key to messages sent after a [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object)
so that a new random symmetric key can be derived, so that a new random symmetric key can be derived,
in a similar fashion to [Double-Ratchet](https://signal.org/docs/specifications/doubleratchet/). in a similar fashion to [Double-Ratchet](https://signal.org/docs/specifications/doubleratchet/).
This can be practically achieved by: This can be practically achieved by:
- keeping the full Handhshake State even after the handshake is complete (*by Noise specification a call to [`Split()`](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) should delete the Handshake State*)
- keeping the full Handhshake State even after the handshake is complete (_by Noise specification a call to [`Split()`](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) should delete the Handshake State_)
- continuing updating the Handshake State by processing every after-handshake exchanged message (i.e. the `payload`) according to the Noise [processing rules](http://www.noiseprotocol.org/noise.html#processing-rules) (i.e. by calling `EncryptAndHash(payload)` and `DecryptAndHash(payload)`); - continuing updating the Handshake State by processing every after-handshake exchanged message (i.e. the `payload`) according to the Noise [processing rules](http://www.noiseprotocol.org/noise.html#processing-rules) (i.e. by calling `EncryptAndHash(payload)` and `DecryptAndHash(payload)`);
- adding to each (or every few) message exchanged in the transfer phase a random ephemeral key `e` and perform Diffie-Hellman operations with the other party's ephemeral/static keys in order to update the underlying CipherState and recover new random inbound/outbound encryption keys by calling [`Split()`](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object). - adding to each (or every few) message exchanged in the transfer phase a random ephemeral key `e` and perform Diffie-Hellman operations with the other party's ephemeral/static keys in order to update the underlying CipherState and recover new random inbound/outbound encryption keys by calling [`Split()`](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object).
@ -236,40 +244,40 @@ TransferPhase:
-> eA, eAeB, eAsB {payload} -> eA, eAeB, eAsB {payload}
<- eB, eAeB, sAeB {payload} <- eB, eAeB, sAeB {payload}
... ...
{}: payload {}: payload
``` ```
### Messages Nametag Derivation ### Messages Nametag Derivation
To reduce metadata leakages and increase devices's anonymity over the p2p network, To reduce metadata leakages and increase devices's anonymity over the p2p network,
[WAKU2-NOISE](./noise.md/#session-states) suggests to use some common secrets `mntsInbound, mntsOutbound` (e.g. `mntsInbound, mntsOutbound = HKDF(h)` [WAKU2-NOISE](./noise.md/#session-states) suggests to use some common secrets `mntsInbound, mntsOutbound` (e.g. `mntsInbound, mntsOutbound = HKDF(h)`
where `h` is the [handshake hash value](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) of the Handshake State at some point of the pairing phase) where `h` is the [handshake hash value](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) of the Handshake State at some point of the pairing phase)
in order to frequently and deterministically change the `messageNametag` of messages exchanged during the pairing and transfer phase - in order to frequently and deterministically change the `messageNametag` of messages exchanged during the pairing and transfer phase -
ideally, at each message exchanged. ideally, at each message exchanged.
Given the proposed construction, Given the proposed construction,
the `mntsInbound` and `mntsOutbound` secrets can be used to iteratively generate the `messageNametag` field of Waku payloads the `mntsInbound` and `mntsOutbound` secrets can be used to iteratively generate the `messageNametag` field of Waku payloads
for inbound and outbound messages, respectively. for inbound and outbound messages, respectively.
The derivation of `messageNametag` should be deterministic only for communicating devices The derivation of `messageNametag` should be deterministic only for communicating devices
and independent from message content, and independent from message content,
otherwise lost messages will prevent computing the next message nametag. otherwise lost messages will prevent computing the next message nametag.
A possible approach consists in computing the `n`-th `messageNametag` as `H( mntsInbound || n)`, A possible approach consists in computing the `n`-th `messageNametag` as `H( mntsInbound || n)`,
where `n` is serialized as `uint64`. where `n` is serialized as `uint64`.
In this way, sender's and recipient's devices In this way, sender's and recipient's devices
can keep updated a buffer of `messageNametag` to sieve can keep updated a buffer of `messageNametag` to sieve
while listening to messages sent over `/{application-name}/{application-version}/wakunoise/1/sessions-{shard-id}/` (i.e., the next 50 not yet seen). while listening to messages sent over `/{application-name}/{application-version}/wakunoise/1/sessions-{shard-id}/` (i.e., the next 50 not yet seen).
They will then be able to further identify if one or more messages were eventually lost They will then be able to further identify if one or more messages were eventually lost
or not-yet-delivered during the communication. or not-yet-delivered during the communication.
This approach brings also the advantage that This approach brings also the advantage that
communicating devices can efficiently identify encrypted messages addressed to them. communicating devices can efficiently identify encrypted messages addressed to them.
We note that since the `ChaChaPoly` cipher used to encrypt messages supports *additional data*, We note that since the `ChaChaPoly` cipher used to encrypt messages supports _additional data_,
an encrypted payload can be further authenticated by passing the `messageNametag` as additional data to the encryption/decryption routine. an encrypted payload can be further authenticated by passing the `messageNametag` as additional data to the encryption/decryption routine.
In this way, an attacker would be unable to craft an authenticated Waku message In this way, an attacker would be unable to craft an authenticated Waku message
even in case the currently used symmetric encryption key is compromised, even in case the currently used symmetric encryption key is compromised,
unless `mntsInbound`, `mntsOutbound` or the `messageNametag` buffer lists were compromised too. unless `mntsInbound`, `mntsOutbound` or the `messageNametag` buffer lists were compromised too.
## Security/Privacy Considerations ## Security/Privacy Considerations
@ -283,61 +291,68 @@ unless `mntsInbound`, `mntsOutbound` or the `messageNametag` buffer lists were c
- Devices `A` and `B` are considered trusted (otherwise the attacker will simply exfiltrate the relevant information from the attacked device). - Devices `A` and `B` are considered trusted (otherwise the attacker will simply exfiltrate the relevant information from the attacked device).
- As common for Noise, we assume that ephemeral keys cannot be compromised, while static keys might be later compromised. However, we enforce in the pairing phase extra security mechanisms (i.e. use of commitments for static keys) that will prevent some attacks possible when ephemeral keys are weak or get compromised. - As common for Noise, we assume that ephemeral keys cannot be compromised, while static keys might be later compromised. However, we enforce in the pairing phase extra security mechanisms (i.e. use of commitments for static keys) that will prevent some attacks possible when ephemeral keys are weak or get compromised.
### Rationale ### Rationale
- The device `B` exposes a commitment to its static key `sB` because: - The device `B` exposes a commitment to its static key `sB` because:
- it can commit to its static key before the authentication code is confirmed without revealing it.
- If the private key of `eB` is weak or gets compromised, an attacker can impersonate `B` by sending in message `c.` to device `A` his own static key and successfully complete the pairing phase. Note that being able to compromise `eB` is not contemplated by our security assumptions. - it can commit to its static key before the authentication code is confirmed without revealing it.
- `B` cannot adaptively choose a static key based on the state of the Noise handshake at the end of message `b.`, i.e. after the authentication code is confirmed. Note that device `B` is trusted in our security assumptions. - If the private key of `eB` is weak or gets compromised, an attacker can impersonate `B` by sending in message `c.` to device `A` his own static key and successfully complete the pairing phase. Note that being able to compromise `eB` is not contemplated by our security assumptions.
- Confirming the authentication code after processing message `b.` will ensure that no Man-in-the-Middle (MitM) can later send a static key different than `sB`. - `B` cannot adaptively choose a static key based on the state of the Noise handshake at the end of message `b.`, i.e. after the authentication code is confirmed. Note that device `B` is trusted in our security assumptions.
- Confirming the authentication code after processing message `b.` will ensure that no Man-in-the-Middle (MitM) can later send a static key different than `sB`.
- The device `A` sends a commitment to its static key `sA` because: - The device `A` sends a commitment to its static key `sA` because:
- it can commit to its static key before the authentication code is confirmed without revealing it.
- `A` cannot adaptively choose a static key based on the state of the Noise handshake at the end of message `b.`, i.e. after the authentication code is confirmed. Note that device `A` is trusted in our security assumptions. - it can commit to its static key before the authentication code is confirmed without revealing it.
- Confirming the authentication code after processing message `b.` will ensure that no MitM can later send a static key different than `sA`. - `A` cannot adaptively choose a static key based on the state of the Noise handshake at the end of message `b.`, i.e. after the authentication code is confirmed. Note that device `A` is trusted in our security assumptions.
- Confirming the authentication code after processing message `b.` will ensure that no MitM can later send a static key different than `sA`.
- The authorization code is shown and has to be confirmed at the end of message `b.` because: - The authorization code is shown and has to be confirmed at the end of message `b.` because:
- an attacker that frontruns device `A` by sending faster his own ephemeral key would be detected before he's able to know device `B` static key `sB`;
- it ensures that no MitM attacks will happen during *the whole* pairing handshake, since commitments to the (later exchanged) device static keys will be implicitly acknowledged by the authorization code confirmation; - an attacker that frontruns device `A` by sending faster his own ephemeral key would be detected before he's able to know device `B` static key `sB`;
- it enables to safely swap the role of handshake initiator and responder (see above); - it ensures that no MitM attacks will happen during _the whole_ pairing handshake, since commitments to the (later exchanged) device static keys will be implicitly acknowledged by the authorization code confirmation;
- it enables to safely swap the role of handshake initiator and responder (see above);
- Device `B` sends his static key first because: - Device `B` sends his static key first because:
- by being the pairing requester, it cannot probe device `A` identity without revealing its own (static key) first. Note that device `B` static key and its commitment can be bound to other cryptographic material (e.g., seed phrase).
- by being the pairing requester, it cannot probe device `A` identity without revealing its own (static key) first. Note that device `B` static key and its commitment can be bound to other cryptographic material (e.g., seed phrase).
- Device `B` opens a commitment to its static key at message `c.` because: - Device `B` opens a commitment to its static key at message `c.` because:
- if device `A` replies concluding the handshake according to the protocol, device `B` acknowledges that device `A` correctly received his static key `sB`, since `r` was encrypted under an encryption key derived from the static key `sB` and the genuine (due to the previous `authcode` verification) ephemeral keys `eA` and `eB`.
- if device `A` replies concluding the handshake according to the protocol, device `B` acknowledges that device `A` correctly received his static key `sB`, since `r` was encrypted under an encryption key derived from the static key `sB` and the genuine (due to the previous `authcode` verification) ephemeral keys `eA` and `eB`.
- Device `A` opens a commitment to its static key at message `d.` because: - Device `A` opens a commitment to its static key at message `d.` because:
- if device `B` doesn't abort the pairing, device `A` acknowledges that device `B` correctly received his static key `sA`, since `s` was encrypted under an encryption key derived from the static keys `sA` and `sB` and the genuine (due to the previous `authcode` verification) ephemeral keys `eA` and `eB`. - if device `B` doesn't abort the pairing, device `A` acknowledges that device `B` correctly received his static key `sA`, since `s` was encrypted under an encryption key derived from the static keys `sA` and `sB` and the genuine (due to the previous `authcode` verification) ephemeral keys `eA` and `eB`.
## Application to Noise Sessions ## Application to Noise Sessions
### The N11M session management mechanism ### The N11M session management mechanism
In the [`N11M` session management mechanism](./noise-sessions/noise-sessions.md/#the-n11m-session-management-mechanism), In the [`N11M` session management mechanism](./noise-sessions.md/#the-n11m-session-management-mechanism),
one of Alice's devices is already communicating with one of Bob's devices within an active Noise session, one of Alice's devices is already communicating with one of Bob's devices within an active Noise session,
e.g. after a successful execution of a Noise handshake. e.g. after a successful execution of a Noise handshake.
Alice and Bob would then share some cryptographic key material, Alice and Bob would then share some cryptographic key material,
used to encrypt their communications. used to encrypt their communications.
According to [WAKU2-NOISE-SESSIONS](./noise-sessions/noise-sessions.md) this information consists of: According to [WAKU2-NOISE-SESSIONS](./noise-sessions.md) this information consists of:
- A `session-id` (32 bytes) - A `session-id` (32 bytes)
- Two cipher state `CSOutbound`, `CSInbound`, where each of them contains: - Two cipher state `CSOutbound`, `CSInbound`, where each of them contains:
- an encryption key `k` (2x32bytes) - an encryption key `k` (2x32bytes)
- a nonce `n` (2x8bytes) - a nonce `n` (2x8bytes)
- (optionally) an internal state hash `h` (2x32bytes) - (optionally) an internal state hash `h` (2x32bytes)
for a total of **176 bytes** of information. for a total of **176 bytes** of information.
In a [`N11M`](./noise-sessions/noise-sessions.md/#the-n11m-session-management-mechanism) session mechanism scenario, In a [`N11M`](./noise-sessions.md/#the-n11m-session-management-mechanism) session mechanism scenario,
all (synced) Alice's devices that are communicating with Bob all (synced) Alice's devices that are communicating with Bob
share the same Noise session cryptographic material. share the same Noise session cryptographic material.
Hence, if Alice wishes to add a new device, Hence, if Alice wishes to add a new device,
she must securely transfer a copy of such data from one of her device `A` to a new device `B` in her possession. she must securely transfer a copy of such data from one of her device `A` to a new device `B` in her possession.
In order to do so she can: In order to do so she can:
- pair device `A` with `B` in order to have a Noise session between them;
- pair device `A` with `B` in order to have a Noise session between them;
- securely transfer within such session the 176 bytes serializing the active session with Bob; - securely transfer within such session the 176 bytes serializing the active session with Bob;
- manually instantiate in `B` a Noise session with Bob from the received session serialization. - manually instantiate in `B` a Noise session with Bob from the received session serialization.
@ -348,10 +363,12 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
### Normative ### Normative
- [35/WAKU2-NOISE](./noise.md/#session-states) - [35/WAKU2-NOISE](./noise.md/#session-states)
- [WAKU2-NOISE-SESSIONS](./noise-sessions/noise-sessions.md/) - [WAKU2-NOISE-SESSIONS](./noise-sessions.md)
### Informative ### Informative
- [26/WAKU2-PAYLOAD](https://rfc.vac.dev/spec/35/#abnf) - [26/WAKU2-PAYLOAD](https://rfc.vac.dev/spec/35/#abnf)
- [The Double-Ratchet Algorithm](https://signal.org/docs/specifications/doubleratchet/) - [The Double-Ratchet Algorithm](https://signal.org/docs/specifications/doubleratchet/)
- [The Noise Protocol Framework specifications](http://www.noiseprotocol.org/noise.html) - [The Noise Protocol Framework specifications](http://www.noiseprotocol.org/noise.html)

112
standards/application/noise-sessions.md
View File

@ -1,16 +1,16 @@
--- ---
title: WAKU2-NOISE-SESSIONS title: WAKU2-NOISE-SESSIONS
name: Session Management for Waku Noise name: Session Management for Waku Noise
tags: waku-core-protocol tags: [waku-core-protocol]
editor: Giuseppe <giuseppe@status.im> editor: Giuseppe <giuseppe@status.im>
contributors: contributors:
--- ---
## Introduction ## Introduction
In [WAKU2-NOISE](../noise.md) we defined how Waku messages' payloads can be encrypted using key material derived from key agreements based on the [Noise Protocol Framework](http://www.noiseprotocol.org/noise.html). In [WAKU2-NOISE](./noise.md) we defined how Waku messages' payloads can be encrypted using key material derived from key agreements based on the [Noise Protocol Framework](http://www.noiseprotocol.org/noise.html).
Once two users complete a Noise handshake, Once two users complete a Noise handshake,
an encryption/decryption session - _or a Noise session_ - would be instantiated. an encryption/decryption session - _or a Noise session_ - would be instantiated.
This post provides an overview on how we can possibly implement and manage one or multiple Noise sessions in Waku. This post provides an overview on how we can possibly implement and manage one or multiple Noise sessions in Waku.
@ -19,59 +19,61 @@ This post provides an overview on how we can possibly implement and manage one o
We assume that two users, e.g. Alice and Bob, successfully completed a Noise handshake. We assume that two users, e.g. Alice and Bob, successfully completed a Noise handshake.
Using [Noise terminology]((http://www.noiseprotocol.org/noise.html)), at the end of the handshake they will share: Using [Noise terminology](http://www.noiseprotocol.org/noise.html), at the end of the handshake they will share:
- two _Cipher States_ `CSOutbound` and `CSInbound`, to encrypt and decrypt outbound and inbound messages, respectively; - two _Cipher States_ `CSOutbound` and `CSInbound`, to encrypt and decrypt outbound and inbound messages, respectively;
- a handshake hash value `h`. - a handshake hash value `h`.
As suggested in Noise specifications in regards to [Channel Binding](http://www.noiseprotocol.org/noise.html#channel-binding), As suggested in Noise specifications in regards to [Channel Binding](http://www.noiseprotocol.org/noise.html#channel-binding),
we can identify a Noise session with a `session-id` derived from the handshake hash value `h` shared on completion of a Noise handshake. we can identify a Noise session with a `session-id` derived from the handshake hash value `h` shared on completion of a Noise handshake.
More specifically, when Alice and Bob call [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) in order to derive the two final encryption and decryption Cipher States, More specifically, when Alice and Bob call [Split()](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object) in order to derive the two final encryption and decryption Cipher States,
they further compute `session-id = HKDF(h)` using the supported key derivation function `HKDF`. they further compute `session-id = HKDF(h)` using the supported key derivation function `HKDF`.
Such `session-id` will uniquely identify the Noise cryptographic session instantiated on completion of a Noise handshake, Such `session-id` will uniquely identify the Noise cryptographic session instantiated on completion of a Noise handshake,
which would then consist of the tuple `(session-id, CSOutbound, CSInbound)`. which would then consist of the tuple `(session-id, CSOutbound, CSInbound)`.
For each instantiated Noise session we assume this tuple to be properly persisted, For each instantiated Noise session we assume this tuple to be properly persisted,
since it is required to either retrieve and encrypt/decrypt any further exchanged message. since it is required to either retrieve and encrypt/decrypt any further exchanged message.
Once a Noise session is instantiated, Once a Noise session is instantiated,
any further encrypted message between Alice and Bob within this session is exchanged on a `contentTopic` with name `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`, any further encrypted message between Alice and Bob within this session is exchanged on a `contentTopic` with name `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`,
where `ct-id = Hash(Hash(session-id))` where `ct-id = Hash(Hash(session-id))`
and `/{application-name}/{application-version}/` identifies the application currently employing [WAKU2-NOISE](../noise.md). and `/{application-name}/{application-version}/` identifies the application currently employing [WAKU2-NOISE](./noise.md).
## Session states ## Session states
A Noise session corresponding to a certain `session-id`: A Noise session corresponding to a certain `session-id`:
- is always **active** as long as it is not marked as **stale**.
For an active `session-id`, new messages are published on the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`;
- is marked as **stale** if a [session termination message](../noise.md/#session-termination-message) containing `Hash(session-id)` is published on the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`.
Session information relative to stale sessions MAY be deleted from users' device, unless required for later channel binding purposes.
When a Noise session is marked as stale, it means that one party requested its termination while being online, - is always **active** as long as it is not marked as **stale**.
For an active `session-id`, new messages are published on the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`;
- is marked as **stale** if a [session termination message](./noise.md/#session-termination-message) containing `Hash(session-id)` is published on the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`.
Session information relative to stale sessions MAY be deleted from users' device, unless required for later channel binding purposes.
When a Noise session is marked as stale, it means that one party requested its termination while being online,
since publication of a hash pre-image for `ct-id` is required (i.e. `Hash(session-id)`). since publication of a hash pre-image for `ct-id` is required (i.e. `Hash(session-id)`).
Currently, it is not possible to mark a Noise session as stale when `session-id` is lost or gets corrupted in users' devices. Currently, it is not possible to mark a Noise session as stale when `session-id` is lost or gets corrupted in users' devices.
However, since `session-id` is shared between Alice and Bob, However, since `session-id` is shared between Alice and Bob,
one party MAY decide to mark a Noise session as stale if no message from the other end was received within a certain fixed time window. one party MAY decide to mark a Noise session as stale if no message from the other end was received within a certain fixed time window.
The above mechanism allows a Noise session to be marked as stale either privately or publicly, The above mechanism allows a Noise session to be marked as stale either privately or publicly,
depending if `Hash(session-id)` is sent on `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto` to the other party in encrypted form or not, respectively. depending if `Hash(session-id)` is sent on `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto` to the other party in encrypted form or not, respectively.
When a Noise session is publicly marked as stale, When a Noise session is publicly marked as stale,
network peers MAY discard all [stored](https://rfc.vac.dev/spec/13/) messages addressed to the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`. network peers MAY discard all [stored](https://rfc.vac.dev/spec/13/) messages addressed to the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`.
In this the case and in order for parties to retrieve any eventually delayed message, In this the case and in order for parties to retrieve any eventually delayed message,
peers SHOULD wait a fixed amount of time before discarding stored messages corresponding to a stale Noise session. peers SHOULD wait a fixed amount of time before discarding stored messages corresponding to a stale Noise session.
A stale Noise session cannot be directly marked as active A stale Noise session cannot be directly marked as active
and parties are required to instantiate a new Noise session if they wish to communicate again. and parties are required to instantiate a new Noise session if they wish to communicate again.
However, parties can optionally persist and include the `session-id` corresponding to a stale Noise session in the [prologue information](https://noiseprotocol.org/noise.html#prologue) employed in the Noise handshake they execute to instantiate their new Noise session. However, parties can optionally persist and include the `session-id` corresponding to a stale Noise session in the [prologue information](https://noiseprotocol.org/noise.html#prologue) employed in the Noise handshake they execute to instantiate their new Noise session.
This effectively emulates a mechanism to _"re-activate"_ a stale Noise session by binding it to a newly created active Noise session. This effectively emulates a mechanism to _"re-activate"_ a stale Noise session by binding it to a newly created active Noise session.
In order to reduce users' metadata leakage, it is desirable (as suggested in [WAKU2-NOISE](../noise.md/#after-handshake)) that content topics used for communications change every time a new message is exchanged. In order to reduce users' metadata leakage, it is desirable (as suggested in [WAKU2-NOISE](./noise.md/#after-handshake)) that content topics used for communications change every time a new message is exchanged.
This can be easily realized by employing a key derivation function to compute a new `session-id` from the previously employed one (e.g. `session-id = HKDF(prev-session-id)`), This can be easily realized by employing a key derivation function to compute a new `session-id` from the previously employed one (e.g. `session-id = HKDF(prev-session-id)`),
while keeping the Inbound/outbound Cipher States, the content topic derivation mechanism and the stale mechanism the same as above. while keeping the Inbound/outbound Cipher States, the content topic derivation mechanism and the stale mechanism the same as above.
In this case, when one party sends **and** receives at least one message, In this case, when one party sends **and** receives at least one message,
he SHALL publicly mark as stale all Noise sessions relative to messages exchanged before the earlier of these two send/receive events. he SHALL publicly mark as stale all Noise sessions relative to messages exchanged before the earlier of these two send/receive events.
## Multi-Device support ## Multi-Device support
@ -79,7 +81,7 @@ he SHALL publicly mark as stale all Noise sessions relative to messages exchange
Alice and Bob might possess one or more devices (e.g. laptops, smartphones, etc.) they wish to use to communicate. Alice and Bob might possess one or more devices (e.g. laptops, smartphones, etc.) they wish to use to communicate.
In the following, we assume Alice and Bob to possess $N$ and $M$ devices, respectively. In the following, we assume Alice and Bob to possess $N$ and $M$ devices, respectively.
Since a Noise session contains cryptographic material required to encrypt and decrypt messages exchanged on a pre-defined content topic derived from a `session-id`, Since a Noise session contains cryptographic material required to encrypt and decrypt messages exchanged on a pre-defined content topic derived from a `session-id`,
messages should be encrypted and decrypted within the Noise session instantiated between the currently-in-use sender's and receiver's device. messages should be encrypted and decrypted within the Noise session instantiated between the currently-in-use sender's and receiver's device.
This is achieved through two main supported session management mechanisms that we called `N11M` and `NM`, respectively. This is achieved through two main supported session management mechanisms that we called `N11M` and `NM`, respectively.
@ -90,66 +92,66 @@ In a $N11M$ setting, each party's device shares the same Noise session informati
![](../../images/N11M.png) ![](../../images/N11M.png)
More precisely, once the first Noise session between any of Alice's and Bob's device is instantiated, More precisely, once the first Noise session between any of Alice's and Bob's device is instantiated,
its session information is securely propagated to all other devices, its session information is securely propagated to all other devices,
which then become able to send and receive new messages on the content topic associated to such session. which then become able to send and receive new messages on the content topic associated to such session.
We note, however, that two devices belonging to one party cannot simultaneously send different messages to the other, since only the first message received will be correctly decrypted using the next nonce. We note, however, that two devices belonging to one party cannot simultaneously send different messages to the other, since only the first message received will be correctly decrypted using the next nonce.
The most updated session information between Alice and Bob is propagated in encrypted form to other devices,
The most updated session information between Alice and Bob is propagated in encrypted form to other devices, using previously instantiated Noise sessions.
using previously instantiated Noise sessions.
In particular, all Alice's (resp., Bob's) devices that want to receive such updated session information, are required to have an already instantiated Noise session between them in order to receive it in encrypted form. In particular, all Alice's (resp., Bob's) devices that want to receive such updated session information, are required to have an already instantiated Noise session between them in order to receive it in encrypted form.
The propagated session information corresponds to the latest session information stored on the device currently communicating with (any of the devices of) the other party. The propagated session information corresponds to the latest session information stored on the device currently communicating with (any of the devices of) the other party.
We note that sessions information is propagated only among devices belonging to the same party and not with other party's devices. We note that sessions information is propagated only among devices belonging to the same party and not with other party's devices.
Hence, Alice has no knowledge on the number of devices Bob is using and vice versa. Hence, Alice has no knowledge on the number of devices Bob is using and vice versa.
When any device marks a Noise session between Alice and Bob as stale, When any device marks a Noise session between Alice and Bob as stale,
all other (updated) devices will consider such session as stale all other (updated) devices will consider such session as stale
without publishing the `Hash(session-id)` on the corresponding session content topic. without publishing the `Hash(session-id)` on the corresponding session content topic.
In case a Noise session between two devices belonging to the same party is marked as stale, In case a Noise session between two devices belonging to the same party is marked as stale,
such two devices stop to reciprocally propagate any information regarding Noise sessions instantiated with other parties. such two devices stop to reciprocally propagate any information regarding Noise sessions instantiated with other parties.
As regards security, an attacker that compromises an encrypted message propagating session information, As regards security, an attacker that compromises an encrypted message propagating session information,
might be able to compromise one or multiple messages exchanged within the session such information refers to. might be able to compromise one or multiple messages exchanged within the session such information refers to.
This can be mitigated by adopting techniques similar to the the ones proposed in [WAKU2-NOISE](../noise.md/#after-handshake), This can be mitigated by adopting techniques similar to the the ones proposed in [WAKU2-NOISE](./noise.md/#after-handshake),
where encryption keys are changed every time a new message is exchanged. where encryption keys are changed every time a new message is exchanged.
This session management mechanism is loosely based on the paper ["Multi-Device for Signal"](https://eprint.iacr.org/2019/1363.pdf). This session management mechanism is loosely based on the paper ["Multi-Device for Signal"](https://eprint.iacr.org/2019/1363.pdf).
## The $NM$ session management mechanism ## The $NM$ session management mechanism
In a $NM$ setting, we require all of $N$ Alice's devices to have an active Noise session with each of Bob's $M$ devices, In a $NM$ setting, we require all of $N$ Alice's devices to have an active Noise session with each of Bob's $M$ devices,
for a total of $NM$ concurrently active Noise sessions between Alice and Bob. for a total of $NM$ concurrently active Noise sessions between Alice and Bob.
![](../../images/NM.png) ![](../../images/NM.png)
A message is sent from the currently-in-use sender's device to all recipent's devices, A message is sent from the currently-in-use sender's device to all recipent's devices,
by properly encrypting and sending it to the content topics of each corresponding active Noise session. by properly encrypting and sending it to the content topics of each corresponding active Noise session.
We note that this allows the recipient to receive a message on all his devices simultaneously. We note that this allows the recipient to receive a message on all his devices simultaneously.
However, on the sender side, only the device which effectively sent the message will know its full content. However, on the sender side, only the device which effectively sent the message will know its full content.
If it is required for sent messages to be available on all sender's devices, If it is required for sent messages to be available on all sender's devices,
each pair of sender's devices SHOULD have an active Noise session used for syncing purposes: each pair of sender's devices SHOULD have an active Noise session used for syncing purposes:
this sums up to a total of $N-1$ and $M-1$ extra Noise sessions instantiated on each Alice's and Bob's device, respectively. this sums up to a total of $N-1$ and $M-1$ extra Noise sessions instantiated on each Alice's and Bob's device, respectively.
Thus, if Alice wants to send a message to Bob from one of her $N$ devices, Thus, if Alice wants to send a message to Bob from one of her $N$ devices,
she encrypts and sends her message to each of Bob's $M$ devices she encrypts and sends her message to each of Bob's $M$ devices
(and, eventually, to each of her other $N-1$ devices), (and, eventually, to each of her other $N-1$ devices),
using the appropriate Noise session information. using the appropriate Noise session information.
If one device marks a Noise session as stale, If one device marks a Noise session as stale,
all active sessions instantiated with such device SHOULD be marked as stale as soon as possible. all active sessions instantiated with such device SHOULD be marked as stale as soon as possible.
If the device declaring a stale session does not send a session termination message to all the other party's devices with which has an active session, If the device declaring a stale session does not send a session termination message to all the other party's devices with which has an active session,
the other party SHOULD send a termination message to mark all such Noise sessions as stale. the other party SHOULD send a termination message to mark all such Noise sessions as stale.
This session management mechanism is loosely based on [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/). This session management mechanism is loosely based on [Signal's Sesame Algorithm](https://signal.org/docs/specifications/sesame/).
# References # References
- [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) - [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/)
- [WAKU2-NOISE](../noise.md) - [WAKU2-NOISE](./noise.md)
- [The Noise Protocol Framework](http://www.noiseprotocol.org/noise.html) - [The Noise Protocol Framework](http://www.noiseprotocol.org/noise.html)
- [The Sesame Algorithm: Session Management for Asynchronous Message Encryption](https://signal.org/docs/specifications/sesame/) - [The Sesame Algorithm: Session Management for Asynchronous Message Encryption](https://signal.org/docs/specifications/sesame/)
- ["Multi-Device for Signal"](https://eprint.iacr.org/2019/1363.pdf) - ["Multi-Device for Signal"](https://eprint.iacr.org/2019/1363.pdf)

206
standards/application/noise.md
View File

@ -1,97 +1,96 @@
--- ---
title: WAKU2-NOISE title: WAKU2-NOISE
name: Noise Protocols for Waku Payload Encryption name: Noise Protocols for Waku Payload Encryption
tags: waku-core-protocol tags: [waku-core-protocol]
editor: Giuseppe <giuseppe@status.im> editor: Giuseppe <giuseppe@status.im>
contributors: contributors:
--- ---
This specification describes how payloads of [Waku messages](https://rfc.vac.dev/spec/14/) with [version 2](https://rfc.vac.dev/spec/14/#version2) can be encrypted This specification describes how payloads of [Waku messages](https://rfc.vac.dev/spec/14/) with [version 2](https://rfc.vac.dev/spec/14/#version2) can be encrypted
in order to achieve confidentiality, authenticity, and integrity in order to achieve confidentiality, authenticity, and integrity
as well as some form of identity-hiding on communicating parties. as well as some form of identity-hiding on communicating parties.
This specification extends the functionalities provided by [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/),
This specification extends the functionalities provided by [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/), adding support to modern symmetric encryption primitives
adding support to modern symmetric encryption primitives
and asymmetric key-exchange protocols. and asymmetric key-exchange protocols.
Specifically, it adds support to the [`ChaChaPoly`](https://www.ietf.org/rfc/rfc7539.txt) cipher for symmetric authenticated encryption.
Specifically, it adds support to the [`ChaChaPoly`](https://www.ietf.org/rfc/rfc7539.txt) cipher for symmetric authenticated encryption. It further describes how the [Noise Protocol Framework](http://www.noiseprotocol.org/noise.html) can be used to exchange cryptographic keys and encrypt/decrypt messages
It further describes how the [Noise Protocol Framework](http://www.noiseprotocol.org/noise.html) can be used to exchange cryptographic keys and encrypt/decrypt messages in a way that the latter are authenticated and protected by _strong forward secrecy_.
in a way that the latter are authenticated and protected by *strong forward secrecy*.
This ultimately allows Waku applications to instantiate end-to-end encrypted communication channels with strong conversational security guarantees, This ultimately allows Waku applications to instantiate end-to-end encrypted communication channels with strong conversational security guarantees,
as similarly done by [5/SECURE-TRANSPORT](https://specs.status.im/spec/5) but in a more modular way, as similarly done by [5/SECURE-TRANSPORT](https://specs.status.im/spec/5) but in a more modular way,
adapting key-exchange protocols to the knowledge communicating parties have of each other. adapting key-exchange protocols to the knowledge communicating parties have of each other.
## Design requirements ## Design requirements
- *Confidentiality*: the adversary should not be able to learn what data is being sent from one Waku endpoint to one or several other Waku endpoints. - _Confidentiality_: the adversary should not be able to learn what data is being sent from one Waku endpoint to one or several other Waku endpoints. - _Strong forward secrecy_: an active adversary cannot decrypt messages nor infer any information on the employed encryption key,
- *Strong forward secrecy*: an active adversary cannot decrypt messages nor infer any information on the employed encryption key, even in the case he has access to communicating parties' long-term private keys (during or after their communication).
even in the case he has access to communicating parties' long-term private keys (during or after their communication). - _Authenticity_: the adversary should not be able to cause a Waku endpoint to accept messages coming from an endpoint different than their original senders.
- *Authenticity*: the adversary should not be able to cause a Waku endpoint to accept messages coming from an endpoint different than their original senders. - _Integrity_: the adversary should not be able to cause a Waku endpoint to accept data that has been tampered with.
- *Integrity*: the adversary should not be able to cause a Waku endpoint to accept data that has been tampered with. - _Identity-hiding_: once a secure communication channel is established,
- *Identity-hiding*: once a secure communication channel is established, a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient.
a passive adversary should not be able to link exchanged encrypted messages to their corresponding sender and recipient.
## Supported Cryptographic Protocols ## Supported Cryptographic Protocols
### Noise Protocols ### Noise Protocols
Two parties executing a Noise protocol exchange one or more [*handshake messages*](http://www.noiseprotocol.org/noise.html#message-format) and/or [*transport messages*](http://www.noiseprotocol.org/noise.html#message-format). Two parties executing a Noise protocol exchange one or more [_handshake messages_](http://www.noiseprotocol.org/noise.html#message-format) and/or [_transport messages_](http://www.noiseprotocol.org/noise.html#message-format).
A Noise protocol consists of one or more Noise handshakes. A Noise protocol consists of one or more Noise handshakes.
During a Noise handshake, two parties exchange multiple handshake messages. During a Noise handshake, two parties exchange multiple handshake messages.
A handshake message contains *ephemeral keys* and/or *static keys* from one of the parties A handshake message contains _ephemeral keys_ and/or _static keys_ from one of the parties
and an encrypted or unencrypted payload that can be used to transmit optional data. and an encrypted or unencrypted payload that can be used to transmit optional data.
These public keys are used to perform a protocol-dependent sequence of Diffie-Hellman operations, These public keys are used to perform a protocol-dependent sequence of Diffie-Hellman operations,
whose results are all hashed into a shared secret key. whose results are all hashed into a shared secret key.
After a handshake is complete, each party will then use the derived shared secret key to send and receive authenticated encrypted transport messages. After a handshake is complete, each party will then use the derived shared secret key to send and receive authenticated encrypted transport messages.
We refer to [Noise protocol framework specifications](http://www.noiseprotocol.org/noise.html#processing-rules) for the full details on how parties shared secret key is derived from each exchanged message. We refer to [Noise protocol framework specifications](http://www.noiseprotocol.org/noise.html#processing-rules) for the full details on how parties shared secret key is derived from each exchanged message.
Four Noise handshakes are currently supported: `K1K1`, `XK1`, `XX`, `XXpsk0`. Their description can be found in [Appendix: Supported Handshakes Description](#Appendix-Supported-Handshake-Description). Four Noise handshakes are currently supported: `K1K1`, `XK1`, `XX`, `XXpsk0`. Their description can be found in [Appendix: Supported Handshakes Description](#Appendix-Supported-Handshake-Description).
These are instantiated combining the following cryptographic primitives: These are instantiated combining the following cryptographic primitives:
- [`Curve25519`](http://www.noiseprotocol.org/noise.html#the-25519-dh-functions) for Diffie-Hellman key-exchanges (32 bytes curve coordinates); - [`Curve25519`](http://www.noiseprotocol.org/noise.html#the-25519-dh-functions) for Diffie-Hellman key-exchanges (32 bytes curve coordinates);
- [`ChaChaPoly`](http://www.noiseprotocol.org/noise.html#the-chachapoly-cipher-functions) for symmetric authenticated encryption (16 bytes authentication tag); - [`ChaChaPoly`](http://www.noiseprotocol.org/noise.html#the-chachapoly-cipher-functions) for symmetric authenticated encryption (16 bytes authentication tag);
- [`SHA256`](http://www.noiseprotocol.org/noise.html#the-sha256-hash-function) hash function used in [`HMAC`](http://www.noiseprotocol.org/noise.html#hash-functions) and [`HKDF`](http://www.noiseprotocol.org/noise.html#hash-functions) keys derivation chains (32 bytes output size); - [`SHA256`](http://www.noiseprotocol.org/noise.html#the-sha256-hash-function) hash function used in [`HMAC`](http://www.noiseprotocol.org/noise.html#hash-functions) and [`HKDF`](http://www.noiseprotocol.org/noise.html#hash-functions) keys derivation chains (32 bytes output size);
#### Content Topics and Message Nametags of Noise Handshake Messages #### Content Topics and Message Nametags of Noise Handshake Messages
We note that all [design requirements](#Design-requirements) on exchanged messages would be satisfied only *after* a supported Noise handshake is completed, We note that all [design requirements](#Design-requirements) on exchanged messages would be satisfied only _after_ a supported Noise handshake is completed,
corresponding to a total of 1 Round Trip Time communication *(1-RTT)*. corresponding to a total of 1 Round Trip Time communication _(1-RTT)_.
In particular, identity-hiding properties can be guaranteed only if the recommendation described in [After-handshake](#After-handshake) are implemented. In particular, identity-hiding properties can be guaranteed only if the recommendation described in [After-handshake](#After-handshake) are implemented.
In the following, we assume that communicating parties reciprocally know an initial [`contentTopic`](https://rfc.vac.dev/spec/14/#wakumessage) In the following, we assume that communicating parties reciprocally know an initial [`contentTopic`](https://rfc.vac.dev/spec/14/#wakumessage)
where they can send/receive the first handshake message(s). where they can send/receive the first handshake message(s).
We further assume that messages sent over a certain `contentTopic` can be efficiently identified by their intended recipients We further assume that messages sent over a certain `contentTopic` can be efficiently identified by their intended recipients
thanks to an arbitrary 16 bytes long `message-nametag` field embedded in the message payload thanks to an arbitrary 16 bytes long `message-nametag` field embedded in the message payload
which is known in advance before messages reception. which is known in advance before messages reception.
The second handshake message MAY be sent/received with a `message-nametag` deterministically derived from the handshake state obtained after processing the first handshake message The second handshake message MAY be sent/received with a `message-nametag` deterministically derived from the handshake state obtained after processing the first handshake message
(using, for example, `HKDF` over the handshake hash value `h`). (using, for example, `HKDF` over the handshake hash value `h`).
This allows This allows
- the recipient to efficiently continue the handshakes started by each initiator; - the recipient to efficiently continue the handshakes started by each initiator;
- the initiators to efficiently associate the recipient's second handshake message to their first handshake message, - the initiators to efficiently associate the recipient's second handshake message to their first handshake message,
However, this does not provide any identity-hiding guarantee to the recipient. However, this does not provide any identity-hiding guarantee to the recipient.
After the second handshake message is correctly received by initiators, the recommendation described in [After-handshake](#After-handshake) SHOULD be implemented to provide full identity-hiding guarantees for both initiator and recipient against passive attackers. After the second handshake message is correctly received by initiators, the recommendation described in [After-handshake](#After-handshake) SHOULD be implemented to provide full identity-hiding guarantees for both initiator and recipient against passive attackers.
### Encryption Primitives ### Encryption Primitives
The symmetric primitives supported are: The symmetric primitives supported are:
- [`ChaChaPoly`](https://www.ietf.org/rfc/rfc7539.txt) for authenticated encryption (16 bytes authentication tag). - [`ChaChaPoly`](https://www.ietf.org/rfc/rfc7539.txt) for authenticated encryption (16 bytes authentication tag).
## Specification ## Specification
When [14/WAKU-MESSAGE version](https://rfc.vac.dev/spec/14/#payload-encryption) is set to 2, When [14/WAKU-MESSAGE version](https://rfc.vac.dev/spec/14/#payload-encryption) is set to 2,
the corresponding `WakuMessage`'s `payload` will encapsulate the two fields `handshake-message` and `transport-message`. the corresponding `WakuMessage`'s `payload` will encapsulate the two fields `handshake-message` and `transport-message`.
The `handshake-message` field MAY contain The `handshake-message` field MAY contain
- a Noise handhshake message (only encrypted/unencrypted public keys). - a Noise handhshake message (only encrypted/unencrypted public keys).
The `transport-message` field MAY contain The `transport-message` field MAY contain
- a Noise handshake message payload (encrypted/unencrypted); - a Noise handshake message payload (encrypted/unencrypted);
- a Noise transport message; - a Noise transport message;
- a `ChaChaPoly` ciphertext. - a `ChaChaPoly` ciphertext.
@ -100,27 +99,20 @@ When a `transport-message` encodes a `ChaChaPoly` ciphertext, the corresponding
The following fields are concatenated to form the `payload` field: The following fields are concatenated to form the `payload` field:
- `message-nametag`: an arbitrary identifier for the Waku message (16 byte). - `message-nametag`: an arbitrary identifier for the Waku message (16 byte).
If the underlying encryption primitive supports it, the contents of this field SHOULD be passed as additional data to the encryption and decryption routines. If the underlying encryption primitive supports it, the contents of this field SHOULD be passed as additional data to the encryption and decryption routines.
- `protocol-id`: identifies the protocol or primitive in use (1 byte). - `protocol-id`: identifies the protocol or primitive in use (1 byte).
Supported values are: Supported values are: - `0`: protocol specification omitted (set for [after-handshake](#After-handshake) messages); - `10`: Noise protocol `Noise_K1K1_25519_ChaChaPoly_SHA256`; - `11`: Noise protocol `Noise_XK1_25519_ChaChaPoly_SHA256`; - `12`: Noise protocol `Noise_XX_25519_ChaChaPoly_SHA256`; - `13`: Noise protocol `Noise_XXpsk0_25519_ChaChaPoly_SHA256`; - `30`: `ChaChaPoly` symmetric encryption.
- `0`: protocol specification omitted (set for [after-handshake](#After-handshake) messages); - `handshake-message-len`: the length in bytes of the Noise handshake message (1 byte).
- `10`: Noise protocol `Noise_K1K1_25519_ChaChaPoly_SHA256`; If `protocol-id` is not equal to `0`, `10`, `11`, `12`, `13`, this field MUST be set to `0`;
- `11`: Noise protocol `Noise_XK1_25519_ChaChaPoly_SHA256`; - `handshake-message`: the Noise handshake message (`handshake-message-len` bytes).
- `12`: Noise protocol `Noise_XX_25519_ChaChaPoly_SHA256`; If `handshake-message-len` is not `0`,
- `13`: Noise protocol `Noise_XXpsk0_25519_ChaChaPoly_SHA256`; it contains the concatenation of one or more Noise Diffie-Hellman ephemeral or static keys
- `30`: `ChaChaPoly` symmetric encryption. encoded as in [Public Keys Encoding](#Public-Keys-Encoding);
- `handshake-message-len`: the length in bytes of the Noise handshake message (1 byte). - `transport-message-len`: the length in bytes of `transport-message` (8 bytes, stored in Little-Endian);
If `protocol-id` is not equal to `0`, `10`, `11`, `12`, `13`, this field MUST be set to `0`; - `transport-message`: the transport message (`transport-message-len` bytes);
- `handshake-message`: the Noise handshake message (`handshake-message-len` bytes). Only during a Noise handshake, this field would contain the Noise handshake message payload.
If `handshake-message-len` is not `0`, The symmetric encryption authentication data for `transport-message`, when present, is appended at the end of `transport-message` (16 bytes).
it contains the concatenation of one or more Noise Diffie-Hellman ephemeral or static keys
encoded as in [Public Keys Encoding](#Public-Keys-Encoding);
- `transport-message-len`: the length in bytes of `transport-message` (8 bytes, stored in Little-Endian);
- `transport-message`: the transport message (`transport-message-len` bytes);
Only during a Noise handshake, this field would contain the Noise handshake message payload.
The symmetric encryption authentication data for `transport-message`, when present, is appended at the end of `transport-message` (16 bytes).
### ABNF ### ABNF
@ -142,7 +134,7 @@ handshake-message = *OCTET
; contains the size of transport-message ; contains the size of transport-message
transport-message-len = *OCTET transport-message-len = *OCTET
; contains the transport message, eventually encrypted. ; contains the transport message, eventually encrypted.
; If encrypted, authentication data is appended ; If encrypted, authentication data is appended
transport-message = *OCTET transport-message = *OCTET
@ -152,117 +144,115 @@ payload = message-nametag protocol-id handshake-message-len handshake-message t
### Protocol Payload Format ### Protocol Payload Format
Based on the specified `protocol-id`, Based on the specified `protocol-id`,
the Waku message `payload` field will encode different types of protocol-dependent messages. the Waku message `payload` field will encode different types of protocol-dependent messages.
In particular, if `protocol-id` is In particular, if `protocol-id` is
- `0`: payload encodes an [after-handshake](#After-handshake) message.
- `handshake-message-len` MAY be 0;
- `transport-message` contains the Noise transport message;
- `10`,`11`,`12`,`13`: payload encodes a supported Noise handshake message.
- `transport-message` contains the Noise transport message;
- `30`: payload encapsulate a `ChaChaPoly` ciphertext `ct`.
- `handshake-message-len` is set to `0`;
- `transport-message` contains the concatenation of the encryption nonce (12 bytes) followed by the ciphertext `ct` and the authentication data for `ct` (16 bytes);
- `transport-message-len` is set accordingly to `transport-message` length;
- `0`: payload encodes an [after-handshake](#After-handshake) message.
- `handshake-message-len` MAY be 0;
- `transport-message` contains the Noise transport message;
- `10`,`11`,`12`,`13`: payload encodes a supported Noise handshake message.
- `transport-message` contains the Noise transport message;
- `30`: payload encapsulate a `ChaChaPoly` ciphertext `ct`.
- `handshake-message-len` is set to `0`;
- `transport-message` contains the concatenation of the encryption nonce (12 bytes) followed by the ciphertext `ct` and the authentication data for `ct` (16 bytes);
- `transport-message-len` is set accordingly to `transport-message` length;
### Public Keys Serialization ### Public Keys Serialization
Diffie-Hellman public keys can be trasmitted in clear Diffie-Hellman public keys can be trasmitted in clear
or in encrypted form (cf. [`WriteMessage`](http://www.noiseprotocol.org/noise.html#the-handshakestate-object)) with authentication data attached. or in encrypted form (cf. [`WriteMessage`](http://www.noiseprotocol.org/noise.html#the-handshakestate-object)) with authentication data attached.
To distinguish between these two cases, public keys are serialized as the concatenation of the following three fields: To distinguish between these two cases, public keys are serialized as the concatenation of the following three fields:
- `flag`: - `flag`:
is equal to `1` if the public key is encrypted; is equal to `1` if the public key is encrypted;
`0` otherwise (1 byte); `0` otherwise (1 byte);
- `pk`: - `pk`:
if `flag = 0`, it contains an encoding of the X coordinate of the public key. if `flag = 0`, it contains an encoding of the X coordinate of the public key.
If `flag = 1`, it contains a symmetric encryption of an encoding of the X coordinate of the public key, followed by encryption's authentication data; If `flag = 1`, it contains a symmetric encryption of an encoding of the X coordinate of the public key, followed by encryption's authentication data;
The corresponding serialization is obtained as `flag pk`. The corresponding serialization is obtained as `flag pk`.
As regards the underlying supported [cryptographic primitives](#Cryptographic-primitives): As regards the underlying supported [cryptographic primitives](#Cryptographic-primitives):
- `Curve25519` public keys X coordinates are encoded in little-endian as 32 bytes arrays; - `Curve25519` public keys X coordinates are encoded in little-endian as 32 bytes arrays;
- `ChaChaPoly` authentication data consists of 16 bytes - `ChaChaPoly` authentication data consists of 16 bytes
(nonces are implicitely defined by Noise [processing rules](http://www.noiseprotocol.org/noise.html#processing-rules)). (nonces are implicitely defined by Noise [processing rules](http://www.noiseprotocol.org/noise.html#processing-rules)).
In all supported Noise protocols, In all supported Noise protocols,
parties' static public keys are transmitted encrypted (cf. [`EncryptAndHash`](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object)), parties' static public keys are transmitted encrypted (cf. [`EncryptAndHash`](http://www.noiseprotocol.org/noise.html#the-symmetricstate-object)),
while ephemeral keys MAY be encrypted after a handshake is complete. while ephemeral keys MAY be encrypted after a handshake is complete.
### Padding ### Padding
To prevent some metadata leakage, To prevent some metadata leakage,
encrypted transport messages SHOULD be padded before encryption. encrypted transport messages SHOULD be padded before encryption.
It is therefore recommended to right pad transport messages using [RFC2630](https://datatracker.ietf.org/doc/html/rfc2630#section-6.3) so that their final length is a multiple of 248 bytes. It is therefore recommended to right pad transport messages using [RFC2630](https://datatracker.ietf.org/doc/html/rfc2630#section-6.3) so that their final length is a multiple of 248 bytes.
## After-handshake ## After-handshake
During the initial 1-RTT communication, During the initial 1-RTT communication,
handshake messages [might be linked](#Content-Topics-and-Message-Nametags-of-Noise-Handshake-Messages), handshake messages [might be linked](#Content-Topics-and-Message-Nametags-of-Noise-Handshake-Messages),
depending on the `message-nametag` derivation rule implemented, depending on the `message-nametag` derivation rule implemented,
to the respective parties through the `contentTopic` and `message-nametag` fields employed for such communication. to the respective parties through the `contentTopic` and `message-nametag` fields employed for such communication.
After a handshake is completed, After a handshake is completed,
parties MAY derive from their shared secret key (preferably using `HKDF`) parties MAY derive from their shared secret key (preferably using `HKDF`)
two random `nametag-secret-outbound` and `nametag-secret-inbound` values used to deterministically derive two random `nametag-secret-outbound` and `nametag-secret-inbound` values used to deterministically derive
two arbitrary-long ordered lists of `message-nametag` two arbitrary-long ordered lists of `message-nametag`
used to indentify outbound and inbound messages, respectively used to indentify outbound and inbound messages, respectively
(e.g. the `n`-th inbound `message-nametag` MAY be computed as `HKDF(nametag-secret-inbound || n)`). (e.g. the `n`-th inbound `message-nametag` MAY be computed as `HKDF(nametag-secret-inbound || n)`).
This allows communicating parties to efficiently identify messages addressed to them sent over a certain `contentTopic` This allows communicating parties to efficiently identify messages addressed to them sent over a certain `contentTopic`
and thus minimize the number of trial decryptions. and thus minimize the number of trial decryptions.
When communicating, When communicating,
parties SHOULD set `protocol-id` to `0` parties SHOULD set `protocol-id` to `0`
to reduce metadata leakages and indicate that the message is an *after-handshake* message. to reduce metadata leakages and indicate that the message is an _after-handshake_ message.
Each party SHOULD attach an (unencrypted) ephemeral key in `handshake-message` to every message sent. Each party SHOULD attach an (unencrypted) ephemeral key in `handshake-message` to every message sent.
According to [Noise processing rules](http://www.noiseprotocol.org/noise.html#processing-rules), According to [Noise processing rules](http://www.noiseprotocol.org/noise.html#processing-rules),
this allows updates to the shared secret key this allows updates to the shared secret key
by hashing the result of an ephemeral-ephemeral Diffie-Hellman exchange every 1-RTT communication. by hashing the result of an ephemeral-ephemeral Diffie-Hellman exchange every 1-RTT communication.
## Backward Support for Symmetric/Asymmetric Encryption ## Backward Support for Symmetric/Asymmetric Encryption
It is possible to have backward compatibility to symmetric/asymmetric encryption primitives from [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/), It is possible to have backward compatibility to symmetric/asymmetric encryption primitives from [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/),
effectively encapsulating payload encryption [14/WAKU-MESSAGE version 1](https://rfc.vac.dev/spec/14/#version1) in [version 2](https://rfc.vac.dev/spec/14/#version2). effectively encapsulating payload encryption [14/WAKU-MESSAGE version 1](https://rfc.vac.dev/spec/14/#version1) in [version 2](https://rfc.vac.dev/spec/14/#version2).
It suffices to extend the list of supported `protocol-id` to: It suffices to extend the list of supported `protocol-id` to:
- `254`: AES-256-GCM symmetric encryption; - `254`: AES-256-GCM symmetric encryption;
- `255`: ECIES asymmetric encryption. - `255`: ECIES asymmetric encryption.
and set the `transport-message` field to the [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26) `data` field, whenever these `protocol-id` values are set. and set the `transport-message` field to the [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26) `data` field, whenever these `protocol-id` values are set.
Namely, if `protocol-id = 254, 255` then: Namely, if `protocol-id = 254, 255` then:
- `message-nametag`: is empty; - `message-nametag`: is empty;
- `handshake-message-len`: is set to `0`; - `handshake-message-len`: is set to `0`;
- `handshake-message`: is empty; - `handshake-message`: is empty;
- `transport-message`: contains the [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/) `data` field (AES-256-GCM or ECIES, depending on `protocol-id`); - `transport-message`: contains the [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/) `data` field (AES-256-GCM or ECIES, depending on `protocol-id`);
- `transport-message-len` is set accordingly to `transport-message` length; - `transport-message-len` is set accordingly to `transport-message` length;
When a `transport-message` corresponding to `protocol-id = 254, 255` is retrieved, When a `transport-message` corresponding to `protocol-id = 254, 255` is retrieved,
it SHOULD be decoded as the `data` field in [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/) specification. it SHOULD be decoded as the `data` field in [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/) specification.
## Appendix: Supported Handshakes Description ## Appendix: Supported Handshakes Description
Supported Noise handshakes address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:
Supported Noise handshakes address four typical scenarios occurring when an encrypted communication channel between Alice and Bob is going to be created:
- Alice and Bob know each others' static key. - Alice and Bob know each others' static key.
- Alice knows Bob's static key; - Alice knows Bob's static key;
- Alice and Bob share no key material and they don't know each others' static key. - Alice and Bob share no key material and they don't know each others' static key.
- Alice and Bob share some key material, but they don't know each others' static key. - Alice and Bob share some key material, but they don't know each others' static key.
**Adversarial Model**: an active attacker who compromised one party's static key may lower the identity-hiding security guarantees provided by some handshakes. In our security model we exclude such adversary, but for completeness we report a summary of possible de-anonymization attacks that can be performed by an active attacker. **Adversarial Model**: an active attacker who compromised one party's static key may lower the identity-hiding security guarantees provided by some handshakes. In our security model we exclude such adversary, but for completeness we report a summary of possible de-anonymization attacks that can be performed by an active attacker.
### The `K1K1` Handshake ### The `K1K1` Handshake
If Alice and Bob know each others' static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a `K1K1` handshake. Using [Noise notation](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) *(Alice is on the left)* this can be sketched as: If Alice and Bob know each others' static key (e.g., these are public or were already exchanged in a previous handshake) , they MAY execute a `K1K1` handshake. Using [Noise notation](https://noiseprotocol.org/noise.html#overview-of-handshake-state-machine) _(Alice is on the left)_ this can be sketched as:
``` ```
K1K1: K1K1:
@ -297,10 +287,9 @@ Within this handshake, Alice and Bob reciprocally authenticate their static keys
**Security considerations on identity-hiding (active attacker)**: Alice's static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob's static key against recorded/accepted exchanged handshake messages. **Security considerations on identity-hiding (active attacker)**: Alice's static key is encrypted with forward secrecy to an authenticated party. An active attacker initiating the handshake can check candidates for Bob's static key against recorded/accepted exchanged handshake messages.
### The `XX` and `XXpsk0` Handshakes ### The `XX` and `XXpsk0` Handshakes
If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an `XX` handshake, where each party tran**X**mits to the other its own static key. If Alice is not aware of any static key belonging to Bob (and neither Bob knows anything about Alice), she can execute an `XX` handshake, where each party tran**X**mits to the other its own static key.
The handshake goes as follows: The handshake goes as follows:
@ -313,7 +302,6 @@ The handshake goes as follows:
We note that the main difference with `XK1` is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange. We note that the main difference with `XK1` is that in second step Bob sends to Alice his own static key encrypted with a key obtained from an ephemeral-ephemeral Diffie-Hellman exchange.
This handshake can be slightly changed in case both Alice and Bob pre-shares some secret `psk` which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called `XXpsk0`, goes as follow: This handshake can be slightly changed in case both Alice and Bob pre-shares some secret `psk` which can be used to strengthen their mutual authentication during the handshake execution. One of the resulting protocol, called `XXpsk0`, goes as follow:
``` ```
@ -322,12 +310,11 @@ This handshake can be slightly changed in case both Alice and Bob pre-shares som
<- e, ee, s, es <- e, ee, s, es
-> s, se -> s, se
``` ```
The main difference with `XX` is that Alice's and Bob's static keys, when transmitted, would be encrypted with a key derived from `psk` as well. The main difference with `XX` is that Alice's and Bob's static keys, when transmitted, would be encrypted with a key derived from `psk` as well.
**Security considerations on identity-hiding (active attacker)**: Alice's static key is encrypted with forward secrecy to an authenticated party for both `XX` and `XXpsk0` handshakes. In `XX`, Bob's static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In `XXpsk0`, instead, Bob's secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret `psk` but not through any static key), provided that `psk` was not previously compromised (in such case identity-hiding properties provided by the `XX` handshake applies). **Security considerations on identity-hiding (active attacker)**: Alice's static key is encrypted with forward secrecy to an authenticated party for both `XX` and `XXpsk0` handshakes. In `XX`, Bob's static key is encrypted with forward secrecy but is transmitted to a non-authenticated user which can then be an active attacker. In `XXpsk0`, instead, Bob's secret key is protected by forward secrecy to a partially authenticated party (through the pre-shared secret `psk` but not through any static key), provided that `psk` was not previously compromised (in such case identity-hiding properties provided by the `XX` handshake applies).
## References ## References
1. [5/SECURE-TRANSPORT](https://specs.status.im/spec/5) 1. [5/SECURE-TRANSPORT](https://specs.status.im/spec/5)
@ -339,7 +326,6 @@ The main difference with `XX` is that Alice's and Bob's static keys, when transm
7. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) 7. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)
8. [RFC2630 - Content-encryption Process and padding](https://datatracker.ietf.org/doc/html/rfc2630#section-6.3) 8. [RFC2630 - Content-encryption Process and padding](https://datatracker.ietf.org/doc/html/rfc2630#section-6.3)
## Copyright ## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

11
standards/application/tor-push.md
View File

@ -2,7 +2,7 @@
title: TOR-PUSH title: TOR-PUSH
name: Waku v2 Tor Push name: Waku v2 Tor Push
category: Best Current Practice category: Best Current Practice
tags: waku/application tags: [waku/application]
editor: Daniel Kaiser <danielkaiser@status.im> editor: Daniel Kaiser <danielkaiser@status.im>
contributors: contributors:
--- ---
@ -20,7 +20,6 @@ Note: Waku Tor Push does not have a dedicated protocol identifier.
It uses the same identifier as Waku relay. It uses the same identifier as Waku relay.
This allows Waku relay nodes that are oblivious to Tor Push to process messages received via Tor Push. This allows Waku relay nodes that are oblivious to Tor Push to process messages received via Tor Push.
## Functional Operation ## Functional Operation
In its current version, Waku Tor Push corresponds to [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46) In its current version, Waku Tor Push corresponds to [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46)
@ -37,7 +36,7 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
* [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11) - [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11)
* [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) - [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
* [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46) - [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46)
* [Tor](https://www.torproject.org/) - [Tor](https://www.torproject.org/)

63
standards/core/enr.md
View File

@ -1,7 +1,7 @@
--- ---
title: WAKU2-ENR title: WAKU2-ENR
name: Waku v2 usage of ENR name: Waku v2 usage of ENR
tags: waku/core-protocol tags: [waku/core-protocol]
editor: Franck Royer <franck@status.im> editor: Franck Royer <franck@status.im>
contributors: contributors:
--- ---
@ -31,6 +31,7 @@ Hence, this would only provide a short term solution until another RFC would nee
Moreover, secure websocket involves SSL certificates. Moreover, secure websocket involves SSL certificates.
SSL certificates are only valid for a given domain and ip, so an ENR containing the following information: SSL certificates are only valid for a given domain and ip, so an ENR containing the following information:
- secure websocket port - secure websocket port
- ipv4 fqdn - ipv4 fqdn
- ipv4 address - ipv4 address
@ -44,13 +45,14 @@ The [10/WAKU2](https://rfc.vac.dev/spec/10/) protocol family is built on the [li
Hence, it uses [multiaddr](https://github.com/multiformats/multiaddr) to format network addresses. Hence, it uses [multiaddr](https://github.com/multiformats/multiaddr) to format network addresses.
Directly storing one or several multiaddresses in the ENR would fix the issues listed above: Directly storing one or several multiaddresses in the ENR would fix the issues listed above:
- multiaddr is self-describing and support addresses for any network protocol: - multiaddr is self-describing and support addresses for any network protocol:
No new RFC would be needed to support encoding other transport protocols in an ENR. No new RFC would be needed to support encoding other transport protocols in an ENR.
- multiaddr contains both the host and port information, allowing the ambiguity previously described to be resolved. - multiaddr contains both the host and port information, allowing the ambiguity previously described to be resolved.
## `multiaddrs` ENR key ## `multiaddrs` ENR key
We define a `multiaddrs` key. We define a `multiaddrs` key.
- The value MUST be a list of binary encoded multiaddr prefixed by their size. - The value MUST be a list of binary encoded multiaddr prefixed by their size.
- The size of the multiaddr MUST be encoded in a Big Endian unsigned 16-bit integer. - The size of the multiaddr MUST be encoded in a Big Endian unsigned 16-bit integer.
@ -70,6 +72,7 @@ We define a `multiaddrs` key.
#### Many connection types #### Many connection types
Alice is a node operator, she runs a node that supports inbound connection for the following protocols: Alice is a node operator, she runs a node that supports inbound connection for the following protocols:
- TCP 10101 on `1.2.3.4` - TCP 10101 on `1.2.3.4`
- UDP 20202 on `1.2.3.4` - UDP 20202 on `1.2.3.4`
- TCP 30303 on `1234:5600:101:1::142` - TCP 30303 on `1234:5600:101:1::142`
@ -80,18 +83,19 @@ Alice is a node operator, she runs a node that supports inbound connection for t
Alice SHOULD structure the ENR for her node as follows: Alice SHOULD structure the ENR for her node as follows:
| key | value | | key | value |
|--- |--- | | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tcp` | `10101` | | `tcp` | `10101` |
| `udp` | `20202` | | `udp` | `20202` |
| `tcp6` | `30303` | | `tcp6` | `30303` |
| `udp6` | `40404` | | `udp6` | `40404` |
| `ip` | `1.2.3.4` | | `ip` | `1.2.3.4` |
| `ip6` | `1234:5600:101:1::142` | | `ip6` | `1234:5600:101:1::142` |
| `secp256k1` | Alice's compressed secp256k1 public key, 33 bytes | | `secp256k1` | Alice's compressed secp256k1 public key, 33 bytes |
| `multiaddrs` | <code>len1 &#124; /dns4/example.com/tcp/443/wss &#124; len2 &#124; /dns4/quic.examle.com/tcp/443/quic &#124; len3 &#124; /ip4/1.2.3.4/tcp/55555/p2p/QmRelay </code> | | `multiaddrs` | <code>len1 &#124; /dns4/example.com/tcp/443/wss &#124; len2 &#124; /dns4/quic.examle.com/tcp/443/quic &#124; len3 &#124; /ip4/1.2.3.4/tcp/55555/p2p/QmRelay </code> |
Where: Where:
- `|` is the concatenation operator, - `|` is the concatenation operator,
- `len1` is the length of `/dns4/example.com/tcp/443/wss` byte representation, - `len1` is the length of `/dns4/example.com/tcp/443/wss` byte representation,
- `len2` is the length of `/dns4/quic.examle.com/tcp/443/quic` byte representation. - `len2` is the length of `/dns4/quic.examle.com/tcp/443/quic` byte representation.
@ -100,14 +104,15 @@ Where:
#### Raw TCP only #### Raw TCP only
Bob is a node operator, he runs a node that supports inbound connection for the following protocols: Bob is a node operator, he runs a node that supports inbound connection for the following protocols:
- TCP 10101 on `1.2.3.4` - TCP 10101 on `1.2.3.4`
Bob SHOULD structure the ENR for her node as follows: Bob SHOULD structure the ENR for her node as follows:
| key | value | | key | value |
|--- |--- | | ----------- | ----------------------------------------------- |
| `tcp` | `10101` | | `tcp` | `10101` |
| `ip` | `1.2.3.4` | | `ip` | `1.2.3.4` |
| `secp256k1` | Bob's compressed secp256k1 public key, 33 bytes | | `secp256k1` | Bob's compressed secp256k1 public key, 33 bytes |
Indeed, as Bob's node's connection details can be represented with EIP-778's pre-defined keys only Indeed, as Bob's node's connection details can be represented with EIP-778's pre-defined keys only
@ -124,32 +129,32 @@ In the future, an extension of this RFC could be made to support other elliptic
We define a `waku2` field key: We define a `waku2` field key:
- The value MUST be an 8-bit flag field, - The value MUST be an 8-bit flag field,
where bits set to `1` indicate `true` and bits set to `0` indicate `false` for the relevant flags. where bits set to `1` indicate `true` and bits set to `0` indicate `false` for the relevant flags.
- The flag values already defined are set out below, - The flag values already defined are set out below,
with `bit 7` the most significant bit and `bit 0` the least significant bit. with `bit 7` the most significant bit and `bit 0` the least significant bit.
| bit 7 | bit 6 | bit 5 | bit 4 | bit 3 | bit 2 | bit 1 | bit 0 | | bit 7 | bit 6 | bit 5 | bit 4 | bit 3 | bit 2 | bit 1 | bit 0 |
| --- | --- | --- | --- | --- | --- | --- | --- | | ------- | ------- | ------- | ------- | ----------- | -------- | ------- | ------- |
| `undef` | `undef` | `undef` | `undef` | `lightpush` | `filter` | `store` | `relay` | | `undef` | `undef` | `undef` | `undef` | `lightpush` | `filter` | `store` | `relay` |
- In the scheme above, the flags `lightpush`, `filter`, `store` and `relay` correlates with support for protocols with the same name. - In the scheme above, the flags `lightpush`, `filter`, `store` and `relay` correlates with support for protocols with the same name.
If a protocol is not supported, the corresponding field MUST be set to `false`. If a protocol is not supported, the corresponding field MUST be set to `false`.
Indicating positive support for any specific protocol is OPTIONAL, Indicating positive support for any specific protocol is OPTIONAL,
though it MAY be required by the relevant application or discovery process. though it MAY be required by the relevant application or discovery process.
- Flags marked as `undef` is not yet defined. - Flags marked as `undef` is not yet defined.
These SHOULD be set to `false` by default. These SHOULD be set to `false` by default.
### Usage ### Usage
- A Waku v2 node MAY choose to populate the `waku2` field for enhanced discovery capabilities, - A Waku v2 node MAY choose to populate the `waku2` field for enhanced discovery capabilities,
such as indicating supported protocols. such as indicating supported protocols.
Such a node MAY indicate support for any specific protocol by setting the corresponding flag to `true`. Such a node MAY indicate support for any specific protocol by setting the corresponding flag to `true`.
- Waku v2 nodes that want to participate in [Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) [[4]](#references), however, - Waku v2 nodes that want to participate in [Node Discovery Protocol v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) [[4]](#references), however,
MUST implement the `waku2` key with at least one flag set to `true`. MUST implement the `waku2` key with at least one flag set to `true`.
- Waku v2 nodes that discovered other participants using Discovery v5, - Waku v2 nodes that discovered other participants using Discovery v5,
MUST filter out participant records that do not implement this field or do not have at least one flag set to `true`. MUST filter out participant records that do not implement this field or do not have at least one flag set to `true`.
- In addition, such nodes MAY choose to filter participants on specific flags (such as supported protocols), - In addition, such nodes MAY choose to filter participants on specific flags (such as supported protocols),
or further interpret the `waku2` field as required by the application. or further interpret the `waku2` field as required by the application.
## Copyright ## Copyright

20
standards/core/incentivization.md
View File

@ -2,11 +2,11 @@
title: WAKU2-INCENTIVIZATION title: WAKU2-INCENTIVIZATION
name: Incentivization for Waku Light Protocols name: Incentivization for Waku Light Protocols
category: Standards Track category: Standards Track
tags: tags: [incentivization]
- incentivization
editor: Sergei Tikhomirov <sergei@status.im> editor: Sergei Tikhomirov <sergei@status.im>
contributors: contributors:
--- ---
## Abstract ## Abstract
This document describes an approach to incentivization of Waku request-response protocols. This document describes an approach to incentivization of Waku request-response protocols.
@ -32,18 +32,20 @@ See a [write-up on incentivization](https://github.com/waku-org/research/blob/1e
## Theory / Semantics ## Theory / Semantics
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt). The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
Consider a request-response protocol with two roles: a client and a server. Consider a request-response protocol with two roles: a client and a server.
A server MAY indicate to a client that it expects certain eligibility criteria to be met. A server MAY indicate to a client that it expects certain eligibility criteria to be met.
In that case, a client MUST provide a valid eligibility proof as part of its request. In that case, a client MUST provide a valid eligibility proof as part of its request.
Forms of eligibility proofs include: Forms of eligibility proofs include:
- Proof of payment: for paid non-authenticated requests. A proof of payment, in turn, may also take different forms, such as a transaction hash or a ZK-proof. In order to interpret a proof of payment, the server needs information about its type. - Proof of payment: for paid non-authenticated requests. A proof of payment, in turn, may also take different forms, such as a transaction hash or a ZK-proof. In order to interpret a proof of payment, the server needs information about its type.
- Proof of membership: for services for a predefined group of users. An example use case: an application developer pays in bulk for their users' requests. A client then prove that they belong to the user set of that application. A similar concept (RLN) is used in Waku Relay for spam prevention. - Proof of membership: for services for a predefined group of users. An example use case: an application developer pays in bulk for their users' requests. A client then prove that they belong to the user set of that application. A similar concept (RLN) is used in Waku Relay for spam prevention.
- Service credential: a proof of membership in a set of clients who have prepaid for the service (which may be considered a special case of proof of membership). - Service credential: a proof of membership in a set of clients who have prepaid for the service (which may be considered a special case of proof of membership).
Upon a receiving a request: Upon a receiving a request:
- the server SHOULD check if the eligibility proof is included and valid; - the server SHOULD check if the eligibility proof is included and valid;
- if that proof is absent or invalid, the server SHOULD send back response with a corresponding error code and error description; - if that proof is absent or invalid, the server SHOULD send back response with a corresponding error code and error description;
- if the proof is valid, the server SHOULD send back the response that the client has requested. - if the proof is valid, the server SHOULD send back the response that the client has requested.
@ -85,10 +87,12 @@ A Store server responds with a list of messages that pass the user's filter.
See [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) for the definitions of `HistoryQuery` and `HistoryResponse`. See [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) for the definitions of `HistoryQuery` and `HistoryResponse`.
The PoC Store incentivization makes the following simplifying assumptions: The PoC Store incentivization makes the following simplifying assumptions:
- the client knows the server's on-chain address `A`; - the client knows the server's on-chain address `A`;
- the client and the server have agreed on a constant price `r` per hour of message history. - the client and the server have agreed on a constant price `r` per hour of message history.
To query messages from a period of length `t`, the client: To query messages from a period of length `t`, the client:
1. calculates the total price `p` as `p = r * t`; 1. calculates the total price `p` as `p = r * t`;
2. pays `p` to the server's address `A` with an on-chain transaction; 2. pays `p` to the server's address `A` with an on-chain transaction;
3. waits until the transaction is confirmed with identifier `txid`; 3. waits until the transaction is confirmed with identifier `txid`;
@ -98,6 +102,7 @@ It is the server's responsibility to keep track of the `txid`s from prior reques
Note that `txid` may not always be practical as proof of payment due to on-chain confirmation latency. Note that `txid` may not always be practical as proof of payment due to on-chain confirmation latency.
To address this issue, future versions of the protocol may involve: To address this issue, future versions of the protocol may involve:
- paying for multiple requests in one transaction; - paying for multiple requests in one transaction;
- using faster (likely L2-based) payment mechanisms. - using faster (likely L2-based) payment mechanisms.
@ -106,6 +111,7 @@ To address this issue, future versions of the protocol may involve:
#### Request #### Request
We extend `HistoryQuery` to include an eligibility proof: We extend `HistoryQuery` to include an eligibility proof:
```protobuf ```protobuf
message HistoryQuery { message HistoryQuery {
// the first field is reserved for future use // the first field is reserved for future use
@ -118,6 +124,7 @@ message HistoryQuery {
``` ```
An example of usage with `txid` as a proof of payment: An example of usage with `txid` as a proof of payment:
```protobuf ```protobuf
HistoryQuery history_query { HistoryQuery history_query {
// the first field is reserved for future use // the first field is reserved for future use
@ -136,6 +143,7 @@ HistoryQuery history_query {
#### Response #### Response
We extend the `HistoryResponse` to indicate the eligibility status: We extend the `HistoryResponse` to indicate the eligibility status:
```protobuf ```protobuf
message HistoryResponse { message HistoryResponse {
// the first field is reserved for future use // the first field is reserved for future use
@ -152,6 +160,7 @@ message HistoryResponse {
``` ```
Example of a response if the client is eligible: Example of a response if the client is eligible:
```protobuf ```protobuf
HistoryResponse response_example { HistoryResponse response_example {
messages: [message_1, message_2] messages: [message_1, message_2]
@ -165,6 +174,7 @@ HistoryResponse response_example {
``` ```
Example of a response if the client is not eligible: Example of a response if the client is not eligible:
```protobuf ```protobuf
HistoryResponse response_example { HistoryResponse response_example {
messages: [] // no messages sent to non-eligible clients messages: [] // no messages sent to non-eligible clients
@ -191,6 +201,7 @@ Additionally, the feasibility of paying for each query is hindered by on-chain f
We will address these challenges as the specification evolves alongside the corresponding PoC implementation. We will address these challenges as the specification evolves alongside the corresponding PoC implementation.
The following ideas will be explored: The following ideas will be explored:
- Batch Payment: instead of paying for an individual query, the client would make a consolidated payment for multiple messages. - Batch Payment: instead of paying for an individual query, the client would make a consolidated payment for multiple messages.
- Price Negotiation: rather than receiving prices off-band, the client would engage in negotiation with the server to determine costs. - Price Negotiation: rather than receiving prices off-band, the client would engage in negotiation with the server to determine costs.
- Dynamic Pricing: the price per message would be variable, based on the total size (in bytes) of all received messages. - Dynamic Pricing: the price per message would be variable, based on the total size (in bytes) of all received messages.
@ -203,16 +214,19 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
### normative ### normative
- A high-level [incentivization outline](https://github.com/waku-org/research/blob/master/incentivization.md) - A high-level [incentivization outline](https://github.com/waku-org/research/blob/master/incentivization.md)
- [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) (for Store-specific sections) - [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) (for Store-specific sections)
### informative ### informative
RFCs of request-response protocols: RFCs of request-response protocols:
- [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12) - [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12)
- [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) - [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/)
- [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) - [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/)
RFCs of Relay and RLN-Relay: RFCs of Relay and RLN-Relay:
- [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11) - [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11)
- [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17) - [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17)

95
standards/core/lightpush.md Normal file
View File

@ -0,0 +1,95 @@
---
title: WAKU-LIGHTPUSH
name: Waku Light Push
editor: Zoltan Nagy <zoltan@status.im>
contributors:
- Hanno Cornelius <hanno@status.im>
- Daniel Kaiser <danielkaiser@status.im>
- Oskar Thorén <oskarth@titanproxy.com>
---
previous version: `/vac/waku/lightpush/2.0.0-beta1` [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/waku/standards/core/19/lightpush)
---
**Protocol identifier**: `/vac/waku/lightpush/3.0.0`
## Motivation and Goals
Light nodes with short connection windows and limited bandwidth wish to push messages to other nodes in the Waku network to request message services.<br>
A common use case is to request that the service node publish the message to an `11/WAKU2-RELAY` pubsub-topic.
Additionally, there is sometimes a need for confirmation that a message has been received "by the network"
(here, at least one node).
`WAKU-LIGHTPUSH` is a request/response protocol for this.
## Payloads
```protobuf
syntax = "proto3";
message LightPushRequest {
string request_id = 1;
// 10 Reserved for future `request_type`. Currently, RELAY is the only available service.
optional string pubsub_topic = 20;
WakuMessage message = 21;
}
message LightPushResponse {
string request_id = 1;
uint32 status_code = 10; // non zero in case of failure, see appendix
optional string status_desc = 11;
optional uint32 relay_peer_count = 12; // number of peers, message is successfully relayed to
}
```
### Message Relaying
Nodes that respond to `LightPushRequest` SHOULD
- either relay the encapsulated message via [11/WAKU2-RELAY](https://rfc.vac.dev/waku/standards/core/11/relay) protocol on the specified `pubsub_topic`<br>
- or perform another requested service.
`Services beyond [11/WAKU2-RELAY](https://rfc.vac.dev/waku/standards/core/11/relay) are as yet underdefined.`
Depending on the network configuration, the lightpush client may not need to provide `pubsub_topic` ([WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)).<br>
If the node is unable to perform the request for some reason, it SHOULD return an error code in `LightPushResponse`.<br>
Once the relay is successful, the `relay_peer_count` will indicate the number of peers that the node has managed to relay the message to. It's important to note that this number may vary depending on the node subscriptions and support for the requested pubsub_topic. The client can use this information to either consider the relay as successful or take further action, such as switching to a lightpush service peer with better connectivity.<br>
>The field `relay_peer_count` may not be present or has the value zero in case of error or in other future use cases, where no relay is involved.
### Examples of possible error codes
| Result | Code | Note |
|--------|------|------|
| SUCCESS | 200 | Successfull push, response's relay_peer_count holds the number of peers the message is pushed. |
| BAD_REQUEST | 400 | Wrong request payload. |
| PAYLOAD_TOO_LARGE | 413 | Message exceeds certain size limit, it can depend on network configuration, see status_desc for details. |
| UNSUPPORTED_PUBSUB_TOPIC | 421 | Requested push on pubsub_topic is not possible as the service node does not support it. |
| TOO_MANY_REQUESTS | 429 | DOS protection prevented this request as the current request exceeds the configured request rate. |
| INTERNAL_SERVER_ERROR | 500 | status_desc holds explanation of the error. |
| NO_PEERS_TO_RELAY | 503 | Lightpush service is not available as the node has no relay peers. |
> The list of error codes is not complete and can be extended in the future.
## Security Considerations
Since this can introduce an amplification factor, it is RECOMMENDED for the node relaying to the rest of the network to take extra precautions.
Therefore Waku applies or will apply:
- DOS protection through request rate limitation on the service itself.
- message rate limiting via [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/waku/standards/core/17/rln-relay), applied via network membership subscription.
> These features are under development.
## Future work
- Add support attaching RLN proof for the message requested to be relayed.
- Add support for other request types.
- Incentivization of the service
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
* [11/WAKU2-RELAY](https://rfc.vac.dev/waku/standards/core/11/relay)
* [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/waku/standards/core/17/rln-relay)

37
standards/core/metadata.md
View File

@ -1,37 +0,0 @@
---
title: WAKU2-METADATA
name: Waku Metadata Protocol
editor: Alvaro Revuelta <alrevuelta@status.im>
contributors:
---
## Metadata Protocol
Waku specifies a req/resp protocol that provides information about the node's medatadata. Such metadata is meant to be used
by the node to decide if a peer is worth connecting or not. The node that makes the request, includes its metadata
so that the receiver is aware of it, without requiring an extra interaction. The parameters are the following:
* `clusterId`: Unique identifier of the cluster that the node is running in.
* `shards`: Shard indexes that the node is subscribed to.
### Protocol id
`/vac/waku/metadata/1.0.0`
### Request
```proto
message WakuMetadataRequest {
optional uint32 cluster_id = 1;
repeated uint32 shards = 2;
}
```
### Response
```proto
message WakuMetadataResponse {
optional uint32 cluster_id = 1;
repeated uint32 shards = 2;
}
```

304
standards/core/network.md
View File

@ -1,304 +0,0 @@
---
title: WAKU2-NETWORK
name: Waku v2 Network
category: Best Current Practice
editor: Hanno Cornelius <hanno@status.im>
contributors:
---
## Abstract
This RFC specifies an opinionated deployment of [10/WAKU2](https://rfc.vac.dev/spec/10/) protocols
to form a coherent and shared decentralized messaging network
that is open-access,
useful for generalized messaging,
privacy-preserving,
scalable and
accessible even to resource-restricted devices.
We'll refer to this opinionated deployment simply as
_the public Waku Network_, _the Waku Network_ or, if the context is clear, _the network_
in the rest of this document.
## Theory / Semantics
### Routing protocol
The Waku Network is built on the [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) routing protocol,
which in turn is an extension of [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/) with spam protection measures.
### Network shards
Traffic in the Waku Network is sharded into eight [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) pubsub topics.
Each pubsub topic is named according to the static shard naming format
defined in [WAKU2-RELAY-SHARDING](../../core/relay-sharding.md)
with:
* `<cluster_id>` set to `1`
* `<shard_number>` occupying the range `0` to `7`.
In other words, the Waku Network is a [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) network
routed on the combination of the eight pubsub topics:
```
/waku/2/rs/1/0
/waku/2/rs/1/1
...
/waku/2/rs/1/7
```
A node MUST use [WAKU-METADATA](./metadata.md) protocol to identify the `<cluster_id>` that every
inbound/outbound peer that attempts to connect supports. In any of the following cases, the node MUST trigger a disconnection:
* [WAKU-METADATA](./metadata.md) dial fails.
* [WAKU-METADATA](./metadata.md) reports an empty `<cluster_id>`.
* [WAKU-METADATA](./metadata.md) reports a `<cluster_id>` different than `1`.
## Roles
There are two distinct roles evident in the network, those of:
1) nodes, and
2) applications.
### Nodes
Nodes are the individual software units
using [10/WAKU2](https://rfc.vac.dev/spec/10/) protocols to form a p2p messaging network.
Nodes, in turn, can participate in a shard as full relayers, i.e. _relay nodes_,
or by running a combination of protocols suitable for resource-restricted environments, i.e. _non-relay nodes_.
Nodes can also provide various services to the network,
such as storing historical messages or protecting the network against spam.
See the section on [default services](#default-services) for more.
#### Relay nodes
Relay nodes MUST follow [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/)
to route messages to other nodes in the network
for any of the pubsub topics [defined as the Waku Network shards](#network-shards).
Relay nodes MAY choose to subscribe to any of these shards,
but MUST be subscribed to at least one defined shard.
Each relay node SHOULD be subscribed to as many shards as it has resources to support.
If a relay node supports an encapsulating application,
it SHOULD be subscribed to all the shards servicing that application.
If resource restrictions prevent a relay node from servicing all shards used by the encapsulating application,
it MAY choose to support some shards as a non-relay node.
#### Bootstrapping and discovery
Nodes MAY use any method to bootstrap connection to the network,
but it is RECOMMENDED that each node retrieves a list of bootstrap peers to connect to using [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459).
Relay nodes SHOULD use [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) to continually discover other peers in the network.
Each relay node MUST encode its supported shards into its discoverable ENR
as described in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#discovery).
The ENR MUST be updated if the set of supported shards change.
A node MAY choose to ignore discovered peers that do not support any of the shards in its own subscribed set.
#### Transports
Relay nodes MUST follow [10/WAKU2](https://rfc.vac.dev/spec/10/) specifications with regards to supporting different transports.
If TCP transport is available, each relay node MUST support it as transport for both dialing and listening.
In addition, a relay node SHOULD support secure websockets for bidirectional communication streams,
for example to allow connections from and to web browser-based clients.
A relay node MAY support unsecure websockets if required by the application or running environment.
#### Default services
For each supported shard,
each relay node SHOULD enable and support the following protocols as a service node:
1. [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12/) to allow resource-restricted peers to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) to allow other peers to request historical messages from this node.
3. [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) to allow resource-restricted peers to request publishing a message to the network on their behalf.
4. [34/WAKU2-PEER-EXCHANGE](../../core/peer-exchange/peer-exchange.md) to allow resource-restricted peers to discover more peers in a resource efficient way.
#### Store service nodes
Each relay node SHOULD support [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) as a store service node,
for each supported shard.
The store SHOULD be configured to retain at least `12` hours of messages per supported shard.
Store service nodes SHOULD only store messages with a valid [`rate_limit_proof`](#message-attributes) attribute.
#### Non-relay nodes
Nodes MAY opt out of relay functionality on any network shard
and instead request services from relay nodes as clients
using any of the defined service protocols:
1. [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12/) to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) to request historical messages matching a specific content filter.
3. [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) to request publishing a message to the network.
4. [34/WAKU2-PEER-EXCHANGE](../../core/peer-exchange/peer-exchange.md) to discover more peers in a resource efficient way.
#### Store client nodes
Nodes MAY request historical messages from [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) service nodes as store clients.
A store client SHOULD discard any messages retrieved from a store service node that do not contain a valid [`rate_limit_proof`](#message-attributes) attribute.
The client MAY consider service nodes returning messages without a valid [`rate_limit_proof`](#message-attributes) attribute as untrustworthy.
The mechanism by which this may happen is currently underdefined.
### Applications
Applications are the higher-layer projects or platforms that make use of the generalized messaging capability of the network.
In other words, an application defines a payload used in the various [10/WAKU2](https://rfc.vac.dev/spec/10/) protocols.
Any participant in an application SHOULD make use of an underlying node in order to communicate on the network.
Applications SHOULD make use of an [autosharding](#autosharding) API
to allow the underlying node to automatically select the target shard on the Waku Network. See the section on [autosharding](#autosharding) for more.
## RLN rate-limiting
The [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) network uses [32/RLN-V1](https://rfc.vac.dev/spec/32/) proofs
to ensure that a pre-agreed rate limit is not exceeded by any publisher.
While the network is under capacity,
individual relayers MAY choose to freely route messages without RLN proofs
up to a discretionary bandwidth limit after which messages without proofs MUST be discarded.
This bandwidth limit SHOULD be enforced using [bandwidth validation mechanism](#free-bandwidth-exceeded) separate from RLN rate-limiting.
This implies that quality of service and reliability is significantly lower for messages without proofs
and at times of high network utilization these messages may not be relayed at all.
### RLN Parameters
For the Waku Network,
the `epoch` is set to `1` second
and the maximum number of messages published per `epoch` is limited to `1` per publisher.
The `max_epoch_gap` is set to `20` seconds,
meaning that validators MUST _reject_ messages with an `epoch` more than 20 seconds into the past or future compared to the validator's own clock.
All nodes, validators and publishers,
SHOULD use Network Time Protocol (NTP) to synchronize their own clocks,
thereby ensuring valid timestamps for proof generation and validation.
### Memberships
Each publisher to the Waku Network SHOULD register an RLN membership
with one of the RLN storage contracts
moderated in the Sepolia registry contract with address [0xF1935b338321013f11068abCafC548A7B0db732C](https://sepolia.etherscan.io/address/0xF1935b338321013f11068abCafC548A7B0db732C#code).
Initial memberships are registered in the Sepolia RLN storage contract with address [0x58322513A35a8f747AF5A385bA14C2AbE602AA59](https://sepolia.etherscan.io/address/0x58322513A35a8f747AF5A385bA14C2AbE602AA59#code).
RLN membership setup and registration MUST follow [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/#setup-and-registration),
with the `staked_fund` set to `0`.
In other words, the Waku Network does not use RLN staking.
### RLN Proofs
Each RLN member MUST generate and attach an RLN proof to every published message
as described in [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/#publishing).
Slashing is not implemented for the Waku Network.
Instead, validators will penalise peers forwarding messages exceeding the rate limit
as specified for [the rate-limiting validation mechanism](#rate-limit-exceeded).
This incentivizes all nodes to validate RLN proofs
and reject messages violating rate limits
in order to continue participating in the network.
## Network traffic
All payload on the Waku Network MUST be encapsulated in a [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/)
with rate limit proof extensions defined for [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/#payloads).
Each message on the Waku Network SHOULD be validated by each relayer,
according to the rules discussed under [message validation](#message-validation).
### Message Attributes
- The mandatory `payload` attribute MUST contain the message data payload as crafted by the application.
- The mandatory `content_topic` attribute MUST specify a string identifier that can be used for content-based filtering. This is also crafted by the application. See [Autosharding](#autosharding) for more on the content topic format.
- The optional `meta` attribute MAY be omitted. If present this will form part of the message uniqueness vector described in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/).
- The optional `version` attribute SHOULD be set to `0`. It MUST be interpreted as `0` if not present.
- The mandatory `timestamp` attribute MUST contain the Unix epoch time at which the message was generated by the application. The value MUST be in nanoseconds. It MAY contain a fudge factor of up to 1 seconds in either direction to improve resistance to timing attacks.
- The optional `ephemeral` attribute MUST be set to `true` if the message should not be persisted by the Waku Network.
- The optional `rate_limit_proof` attribute SHOULD be populated with the RLN proof as set out in [RLN Proofs](#rln-proofs). Messages with this field unpopulated MAY be discarded from the network by relayers. This field MUST be populated if the message should be persisted by the Waku Network.
### Message Size
Any Waku Message published to the network MUST NOT exceed an absolute maximum size of `150` kilobytes.
This limit applies to the entire message after protobuf serialization, including attributes.
It is RECOMMENDED not to exceed an average size of `4` kilobytes for Waku Messages published to the network.
### Message Validation
Relay nodes MUST apply [gossipsub v1.1 validation](https://github.com/libp2p/specs/blob/c96c9ec5909d64fe020d7630f3fd982bc18fd06a/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) to each relayed message and
SHOULD apply all of the rules set out in the section below to determine the validity of a message.
Validation has one of three outcomes,
repeated here from the [gossipsub specification](https://github.com/libp2p/specs/blob/c96c9ec5909d64fe020d7630f3fd982bc18fd06a/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) for ease of reference:
1. Accept - the message is considered valid and it MUST be delivered and forwarded to the network.
2. Reject - the message is considered invalid, MUST be rejected and SHOULD trigger a gossipsub scoring penalty against the transmitting peer.
3. Ignore - the message SHOULD NOT be delivered and forwarded to the network, but this MUST NOT trigger a gossipsub scoring penalty against the transmitting peer.
The following validation rules are defined:
#### Decoding failure
If a message fails to decode as a valid [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/).
the relay node MUST _reject_ the message.
This SHOULD trigger a penalty against the transmitting peer.
#### Invalid timestamp
If a message has a timestamp deviating by more than `20` seconds
either into the past or the future
when compared to the relay node's internal clock,
the relay node MUST _reject_ the message.
This allows for some deviation between internal clocks,
network routing latency and
an optional [fudge factor when timestamping new messages](#message-attributes).
#### Free bandwidth exceeded
If a message contains no RLN proof
and the current bandwidth utilization on the shard the message was published to
equals or exceeds `1` Mbps,
the relay node SHOULD _ignore_ the message.
#### Invalid RLN epoch
If a message contains an RLN proof
and the `epoch` attached to the proof deviates by more than `max_epoch_gap` seconds
from the relay node's own `epoch`,
the relay node MUST _reject_ the message.
`max_epoch_gap` is [set to `20` seconds](#rln-parameters) for the Waku Network.
#### Invalid RLN proof
If a message contains an RLN proof
and the zero-knowledge proof is invalid
according to the verification process described in [32/RLN-V1](https://rfc.vac.dev/spec/32/#verification),
the relay node MUST _ignore_ the message.
#### Rate limit exceeded
If a message contains an RLN proof
and the relay node detects double signaling
according to the verification process described in [32/RLN-V1](https://rfc.vac.dev/spec/32/#verification),
the relay node MUST _reject_ the message
for violating the agreed rate limit of `1` message every `1` second.
This SHOULD trigger a penalty against the transmitting peer.
## Autosharding
Nodes in the Waku Network SHOULD allow encapsulating applications to use autosharding,
as defined in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#automatic-sharding)
by automatically determining the appropriate pubsub topic
from the list [of defined Waku Network shards](#network-shards).
This allows the application to omit the target pubsub topic
when invoking any Waku protocol function.
Applications using autosharding MUST use content topics in the format
defined in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#content-topics-format-for-autosharding)
and SHOULD use the short length format:
```
/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}`
```
When an encapsulating application makes use of autosharding
the underlying node MUST determine the target pubsub topic(s)
from the content topics provided by the application
using the hashing mechanism defined in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#automatic-sharding).
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
* [WAKU2-RELAY-SHARDING](../../core/relay-sharding.md)
* [Peer-exchange](../../core/peer-exchange/peer-exchange.md)
## normative
(TBD)
A list of references that MUST be read to fully understand and/or implement this protocol.
See [RFC3967 Section 1.1](https://datatracker.ietf.org/doc/html/rfc3967#section-1.1).
## informative
(TBD)
A list of additional references.

40
standards/core/peer-exchange.md
View File

@ -2,7 +2,7 @@
title: WAKU2-PEER-EXCHANGE title: WAKU2-PEER-EXCHANGE
name: Waku v2 Peer Exchange name: Waku v2 Peer Exchange
category: Standards Track category: Standards Track
tags: waku/core-protocol tags: [waku/core-protocol]
editor: Daniel Kaiser <danielkaiser@status.im> editor: Daniel Kaiser <danielkaiser@status.im>
contributors: contributors:
--- ---
@ -31,12 +31,12 @@ This protocol SHOULD only be used if [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/
The peer exchange protocol specified in this document is a simple request-response protocol. The peer exchange protocol specified in this document is a simple request-response protocol.
As Figure 1 illustrates, the requesting node sends a request to a peer, which acts as the responder. As Figure 1 illustrates, the requesting node sends a request to a peer, which acts as the responder.
The responder replies with a list of ENRs as specified in [WAKU2-ENR](../enr.md). The responder replies with a list of ENRs as specified in [WAKU2-ENR](./enr.md).
The [multiaddresses](https://docs.libp2p.io/concepts/addressing/) used to connect to the respective peers can be extracted from the ENRs. The [multiaddresses](https://docs.libp2p.io/concepts/addressing/) used to connect to the respective peers can be extracted from the ENRs.
![Figure 1: The responder provides a list of ENRs to the requester. These ENRs contain the information necessary for connecting to the respective peers.](../../images/protocol.svg) ![Figure 1: The responder provides a list of ENRs to the requester. These ENRs contain the information necessary for connecting to the respective peers.](../../images/protocol.svg)
In order to protect its anonymity, the responder MUST NOT provide peers from its actively used peer list as this opens pathways to *Neighbourhood Surveillance* attacks, as described in the In order to protect its anonymity, the responder MUST NOT provide peers from its actively used peer list as this opens pathways to _Neighbourhood Surveillance_ attacks, as described in the
[Security/Privacy Considerations Section](#securityprivacy-considerations). [Security/Privacy Considerations Section](#securityprivacy-considerations).
The responder SHOULD provide a set of peers that has been retrieved using ambient peer discovery methods supporting random sampling, e.g. [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/). The responder SHOULD provide a set of peers that has been retrieved using ambient peer discovery methods supporting random sampling, e.g. [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/).
This both protects the responder's anonymity as well as helps distributing load. This both protects the responder's anonymity as well as helps distributing load.
@ -54,8 +54,8 @@ This document provides recommended choices for the cache size in the [Implementa
Requesters, in the context of the specified peer exchange protocol, SHOULD be resource restricted devices. Requesters, in the context of the specified peer exchange protocol, SHOULD be resource restricted devices.
While any node could technically act as a requester, using the peer exchange protocol comes with two drawbacks: While any node could technically act as a requester, using the peer exchange protocol comes with two drawbacks:
* reducing [anonymity](#securityprivacy-considerations) - reducing [anonymity](#securityprivacy-considerations)
* causing load on responder nodes - causing load on responder nodes
## Wire Format Specification ## Wire Format Specification
@ -81,7 +81,7 @@ message PeerExchangeRPC {
``` ```
The `enr` field contains a Waku ENR as specified in [WAKU2-ENR](../enr.md). The `enr` field contains a Waku ENR as specified in [WAKU2-ENR](./enr.md).
Requesters send a `PeerExchangeQuery` to a peer. Requesters send a `PeerExchangeQuery` to a peer.
Responders SHOULD include a maximum of `num_peers` `PeerInfo` instances into a response. Responders SHOULD include a maximum of `num_peers` `PeerInfo` instances into a response.
@ -101,7 +101,7 @@ depends on the average number of requested peers, which is expected to be the ou
The recommended value for this outbound degree is 6 (see parameter `D` in [29/WAKU2-CONFIG](https://rfc.vac.dev/spec/29/)). The recommended value for this outbound degree is 6 (see parameter `D` in [29/WAKU2-CONFIG](https://rfc.vac.dev/spec/29/)).
It is recommended for the cache to hold at least 10 times as many peers (60). It is recommended for the cache to hold at least 10 times as many peers (60).
The recommended cache size also depends on the number of requesters a responder is expected to serve within a *refresh cycle*. The recommended cache size also depends on the number of requesters a responder is expected to serve within a _refresh cycle_.
A refresh cycle is the time interval in which all peers in the cache are expected to be replaced. A refresh cycle is the time interval in which all peers in the cache are expected to be replaced.
If the number of requests expected per refresh cycle exceeds 600 (10 times the above recommended 60), If the number of requests expected per refresh cycle exceeds 600 (10 times the above recommended 60),
it is recommended to increase the cache size to at least a tenth of that number. it is recommended to increase the cache size to at least a tenth of that number.
@ -116,13 +116,13 @@ We differentiate these implications into the requester and responder side, respe
### Requester ### Requester
With a simple peer exchange protocol, the requester is inherently susceptible to both *neighbourhood surveillance* and *controlled neighbourhood* attacks. With a simple peer exchange protocol, the requester is inherently susceptible to both _neighbourhood surveillance_ and _controlled neighbourhood_ attacks.
To mount a *neighbourhood surveillance* attack, an attacker has to connect to the peers of the victim node. To mount a _neighbourhood surveillance_ attack, an attacker has to connect to the peers of the victim node.
The peer exchange protocol allows a malicious responder to easily get into this position. The peer exchange protocol allows a malicious responder to easily get into this position.
The responder connects to a set of peers and simply returns this set of peers to the requester. The responder connects to a set of peers and simply returns this set of peers to the requester.
The peer exchange protocol also makes it much easier to get into the position required for the *controlled neighbourhood* attack: The peer exchange protocol also makes it much easier to get into the position required for the _controlled neighbourhood_ attack:
A malicious responder provides controlled peers in the response peer list. A malicious responder provides controlled peers in the response peer list.
More on these attacks may be found in our [research log article](https://vac.dev/wakuv2-relay-anon). More on these attacks may be found in our [research log article](https://vac.dev/wakuv2-relay-anon).
@ -131,9 +131,9 @@ As a weak mitigation the requester MAY ask several peers and select a subset of
### Responder ### Responder
Responders that answer with active mesh peers are more vulnerable to a *neighbourhood surveillance* attack. Responders that answer with active mesh peers are more vulnerable to a _neighbourhood surveillance_ attack.
Responding with the set of active mesh peers allows a malicious requester to get into the required position more easily. Responding with the set of active mesh peers allows a malicious requester to get into the required position more easily.
It takes away the first hurdle of the *neighbourhood surveillance* attack: The attacker knows which peers to try to connect to. It takes away the first hurdle of the _neighbourhood surveillance_ attack: The attacker knows which peers to try to connect to.
This increased vulnerability can be avoided by only responding with randomly sampled sets of peers, e.g. by requesting a random peer set via [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/). This increased vulnerability can be avoided by only responding with randomly sampled sets of peers, e.g. by requesting a random peer set via [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/).
(As stated in the [Theory and Protocol Semantics Section](#theory-and-protocol-semantics), (As stated in the [Theory and Protocol Semantics Section](#theory-and-protocol-semantics),
these peer sets SHOULD be retrieved unsolicitedly before receiving requests to achieve faster response times.) these peer sets SHOULD be retrieved unsolicitedly before receiving requests to achieve faster response times.)
@ -146,7 +146,7 @@ Still, frequent queries can tigger the refresh cycle more often. The `seen cache
### Further Considerations ### Further Considerations
The response field contains ENRs as specified in [WAKU2-ENR](../enr.md). The response field contains ENRs as specified in [WAKU2-ENR](./enr.md).
While ENRs contain signatures, they do not violate the [Waku relay no-sign policy](https://rfc.vac.dev/spec/11/#signature-policy)), While ENRs contain signatures, they do not violate the [Waku relay no-sign policy](https://rfc.vac.dev/spec/11/#signature-policy)),
because they are part of the discovery domain and are not propagated in the relay domain. because they are part of the discovery domain and are not propagated in the relay domain.
However, there might still be some form of leakage: However, there might still be some form of leakage:
@ -159,10 +159,10 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
* [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) - [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/)
* [WAKU2-ENR](../enr.md) - [WAKU2-ENR](./enr.md)
* [multiaddress](https://docs.libp2p.io/concepts/addressing/) - [multiaddress](https://docs.libp2p.io/concepts/addressing/)
* [libp2p discovery interface](https://github.com/status-im/nim-libp2p/issues/140) - [libp2p discovery interface](https://github.com/status-im/nim-libp2p/issues/140)
* [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md) - [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
* [29/WAKU2-CONFIG](https://rfc.vac.dev/spec/29/) - [29/WAKU2-CONFIG](https://rfc.vac.dev/spec/29/)
* [Waku relay anonymity](https://vac.dev/wakuv2-relay-anon) - [Waku relay anonymity](https://vac.dev/wakuv2-relay-anon)

113
standards/core/relay-sharding.md
View File

@ -3,10 +3,10 @@ title: RELAY-SHARDING
name: Waku v2 Relay Sharding name: Waku v2 Relay Sharding
status: raw status: raw
category: Standards Track category: Standards Track
tags: waku/core tags: [waku/core]
editor: Daniel Kaiser <danielkaiser@status.im> editor: Daniel Kaiser <danielkaiser@status.im>
contributors: contributors:
- Simon-Pierre Vivier <simvivier@status.im> - Simon-Pierre Vivier <simvivier@status.im>
--- ---
## Abstract ## Abstract
@ -14,7 +14,7 @@ contributors:
This document describes ways of sharding the [Waku relay](https://rfc.vac.dev/spec/11/) topic, This document describes ways of sharding the [Waku relay](https://rfc.vac.dev/spec/11/) topic,
allowing Waku networks to scale in the number of content topics. allowing Waku networks to scale in the number of content topics.
> *Note*: Scaling in the size of a single content topic is out of scope for this document. > _Note_: Scaling in the size of a single content topic is out of scope for this document.
## Background and Motivation ## Background and Motivation
@ -34,7 +34,7 @@ This document also covers discovery of topic shards.
## Named Sharding ## Named Sharding
*Named sharding* offers apps to freely choose pubsub topic names. _Named sharding_ offers apps to freely choose pubsub topic names.
It is RECOMMENDED for App protocols to follow the naming structure detailed in [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/). It is RECOMMENDED for App protocols to follow the naming structure detailed in [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/).
With named sharding, managing discovery falls into the responsibility of apps. With named sharding, managing discovery falls into the responsibility of apps.
@ -44,7 +44,7 @@ From an app protocol point of view, a subscription to a content topic `waku2/xxx
## Static Sharding ## Static Sharding
*Static sharding* offers a set of shards with fixed names. _Static sharding_ offers a set of shards with fixed names.
Assigning content topics to specific shards is up to app protocols, Assigning content topics to specific shards is up to app protocols,
but the discovery of these shards is managed by Waku. but the discovery of these shards is managed by Waku.
@ -56,17 +56,17 @@ A specific shard cluster is either globally available to all apps,
specific for an app protocol, specific for an app protocol,
or reserved for automatic sharding (see next section). or reserved for automatic sharding (see next section).
> *Note:* This leads to $2^16 * 1024 = 2^26$ shards for which Waku manages discovery. > _Note:_ This leads to $2^16 * 1024 = 2^26$ shards for which Waku manages discovery.
App protocols can either choose to use global shards, or app specific shards. App protocols can either choose to use global shards, or app specific shards.
Like the [IANA ports](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml), Like the [IANA ports](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml),
shard clusters are divided into ranges: shard clusters are divided into ranges:
| index (range) | usage | | index (range) | usage |
| --- | --- | | ------------- | -------------------- |
| 0 - 15 | reserved | | 0 - 15 | reserved |
| 16 - 65535| app-defined networks | | 16 - 65535 | app-defined networks |
The informational RFC [WAKU2-RELAY-STATIC-SHARD-ALLOC](../../informational/relay-static-shard-alloc.md) lists the current index allocations. The informational RFC [WAKU2-RELAY-STATIC-SHARD-ALLOC](../../informational/relay-static-shard-alloc.md) lists the current index allocations.
@ -82,9 +82,9 @@ an example for the 2nd shard in the global shard cluster:
`/waku/2/rs/0/2`. `/waku/2/rs/0/2`.
> *Note*: Because *all* shards distribute payload defined in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) via [protocol buffers](https://developers.google.com/protocol-buffers/), > _Note_: Because _all_ shards distribute payload defined in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) via [protocol buffers](https://developers.google.com/protocol-buffers/),
the pubsub topic name does not explicitly add `/proto` to indicate protocol buffer encoding. > the pubsub topic name does not explicitly add `/proto` to indicate protocol buffer encoding.
We use `rs` to indicate these are *relay shard* clusters; further shard types might follow in the future. > We use `rs` to indicate these are _relay shard_ clusters; further shard types might follow in the future.
From an app point of view, a subscription to a content topic `waku2/xxx` on a static shard would look like: From an app point of view, a subscription to a content topic `waku2/xxx` on a static shard would look like:
@ -100,18 +100,18 @@ And for shard 43 of the Status app (which has allocated index 16):
Waku v2 supports the discovery of peers within static shards, Waku v2 supports the discovery of peers within static shards,
so app protocols do not have to implement their own discovery method. so app protocols do not have to implement their own discovery method.
Nodes add information about their shard participation in their [WAKU2-ENR](./enr.md/). Nodes add information about their shard participation in their [WAKU2-ENR](./enr.md).
Having a static shard participation indication as part of the ENR allows nodes Having a static shard participation indication as part of the ENR allows nodes
to discover peers that are part of shards via [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) as well as via DNS. to discover peers that are part of shards via [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) as well as via DNS.
> *Note:* In the current version of this document, > _Note:_ In the current version of this document,
sharding information is directly added to the ENR. > sharding information is directly added to the ENR.
(see Ethereum ENR sharding bit vector [here](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#metadata) > (see Ethereum ENR sharding bit vector [here](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#metadata)
Static relay sharding supports 1024 shards per cluster, leading to a flag field of 128 bytes. > Static relay sharding supports 1024 shards per cluster, leading to a flag field of 128 bytes.
This already takes half (including index and key) of the ENR space of 300 bytes. > This already takes half (including index and key) of the ENR space of 300 bytes.
For this reason, the current specification only supports a single shard cluster per node. > For this reason, the current specification only supports a single shard cluster per node.
In future versions, we will add further (hierarchical) discovery methods. > In future versions, we will add further (hierarchical) discovery methods.
We will update [WAKU2-ENR](./enr.md) accordingly, once this RFC moves forward. > We will update [WAKU2-ENR](./enr.md) accordingly, once this RFC moves forward.
This document specifies two ways of indicating shard cluster participation. This document specifies two ways of indicating shard cluster participation.
The index list SHOULD be used for nodes that participante in fewer than 64 shards, The index list SHOULD be used for nodes that participante in fewer than 64 shards,
@ -122,29 +122,30 @@ Nodes MAY interpret `rs` in such ENRs, but MUST ignore `rsv`.
#### Index List #### Index List
| key | value | | key | value |
|--- |--- | | ---- | ---------------------------------------------------------------------------------------------------------------------- |
| `rs` | <2-byte shard cluster index> &#124; <1-byte length> &#124; <2-byte shard index> &#124; ... &#124; <2-byte shard index> | | `rs` | <2-byte shard cluster index> &#124; <1-byte length> &#124; <2-byte shard index> &#124; ... &#124; <2-byte shard index> |
The ENR key is `rs`. The ENR key is `rs`.
The value is comprised of The value is comprised of
* a two-byte shard cluster index in network byte order, concatenated with
* a one-byte length field holding the number of shards in the given shard cluster, concatenated with - a two-byte shard cluster index in network byte order, concatenated with
* two-byte shard indices in network byte order - a one-byte length field holding the number of shards in the given shard cluster, concatenated with
- two-byte shard indices in network byte order
Example: Example:
| key | value | | key | value |
|--- |--- | | ---- | ------------------------------------------------------- |
| `rs` | 16u16 &#124; 3u8 &#124; 13u16 &#124; 14u16 &#124; 45u16 | | `rs` | 16u16 &#124; 3u8 &#124; 13u16 &#124; 14u16 &#124; 45u16 |
This example node is part of shards `13`, `14`, and `45` in the Status main-net shard cluster (index 16). This example node is part of shards `13`, `14`, and `45` in the Status main-net shard cluster (index 16).
#### Bit Vector #### Bit Vector
| key | value | | key | value |
|--- |--- | | ----- | --------------------------------------------------------- |
| `rsv` | <2-byte shard cluster index> &#124; <128-byte flag field> | | `rsv` | <2-byte shard cluster index> &#124; <128-byte flag field> |
The ENR key is `rsv`. The ENR key is `rsv`.
The value is comprised of a two-byte shard cluster index in network byte order concatenated with a 128-byte wide bit vector. The value is comprised of a two-byte shard cluster index in network byte order concatenated with a 128-byte wide bit vector.
@ -155,9 +156,9 @@ and [this](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/val
Example: Example:
| key | value | | key | value |
|--- |--- | | ----- | -------------------------------------- |
| `rsv` | 16u16 &#124; `0x[...]0000100000003000` | | `rsv` | 16u16 &#124; `0x[...]0000100000003000` |
The `[...]` in the example indicates 120 `0` bytes. The `[...]` in the example indicates 120 `0` bytes.
This example node is part of shards `13`, `14`, and `45` in the Status main-net shard cluster (index 16). This example node is part of shards `13`, `14`, and `45` in the Status main-net shard cluster (index 16).
@ -177,17 +178,19 @@ the `version` (UTF-8 string of N bytes).
The shard to use is the modulo of the hash by the number of shards in the network. The shard to use is the modulo of the hash by the number of shards in the network.
#### Example #### Example
| Field | Value | Hex
|--- |--- |--- | Field | Value | Hex |
| `application` | "myapp"| 0x6d79617070 | ---------------- | ------- | ------------ |
| `version` | "1" | 0x31 | `application` | "myapp" | 0x6d79617070 |
| `network shards`| 8 | 0x8 | `version` | "1" | 0x31 |
| `network shards` | 8 | 0x8 |
- SHA2-256 of `0x6d7961707031` is `0x8e541178adbd8126068c47be6a221d77d64837221893a8e4e53139fb802d4928` - SHA2-256 of `0x6d7961707031` is `0x8e541178adbd8126068c47be6a221d77d64837221893a8e4e53139fb802d4928`
- `0x8e541178adbd8126068c47be6a221d77d64837221893a8e4e53139fb802d4928` MOD `8` equals `0` - `0x8e541178adbd8126068c47be6a221d77d64837221893a8e4e53139fb802d4928` MOD `8` equals `0`
- The shard to use has index 0 - The shard to use has index 0
### Content Topics Format for Autosharding ### Content Topics Format for Autosharding
Content topics MUST follow the format in [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/#content-topic-format). Content topics MUST follow the format in [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/#content-topic-format).
In addition, a generation prefix MAY be added to content topics. In addition, a generation prefix MAY be added to content topics.
When omitted default values are used. When omitted default values are used.
@ -202,11 +205,13 @@ Generation default value is `0`.
- Short length `/myapp/1/mytopic/cbor` - Short length `/myapp/1/mytopic/cbor`
#### Generation #### Generation
The generation number monotonously increases and indirectly refers to the total number of shards of the Waku Network. The generation number monotonously increases and indirectly refers to the total number of shards of the Waku Network.
<!-- Create a new RFC for each generation spec. --> <!-- Create a new RFC for each generation spec. -->
#### Topic Design #### Topic Design
Content topics have 2 purposes: filtering and routing. Content topics have 2 purposes: filtering and routing.
Filtering is done by changing the `{content-topic-name}` field. Filtering is done by changing the `{content-topic-name}` field.
As this part is not hashed, it will not affect routing (shard selection). As this part is not hashed, it will not affect routing (shard selection).
@ -224,7 +229,7 @@ The opposite problem occurs when a mesh only carries multicast groups with very
The current autosharding method does not solve this problem. The current autosharding method does not solve this problem.
> *Note:* Automatic sharding based on network traffic measurements to avoid hot spots in not part of this specification. > _Note:_ Automatic sharding based on network traffic measurements to avoid hot spots in not part of this specification.
#### Discovery #### Discovery
@ -252,14 +257,14 @@ The transition to the second method will be seamless and fully backwards compati
## Security/Privacy Considerations ## Security/Privacy Considerations
See [WAKU2-ADVERSARIAL-MODELS](../../informational/adersarial-models.md), especially the parts on k-anonymity. See [WAKU2-ADVERSARIAL-MODELS](../../informational/adversarial-models.md), especially the parts on k-anonymity.
We will add more on security considerations in future versions of this document. We will add more on security considerations in future versions of this document.
### Receiver Anonymity ### Receiver Anonymity
The strength of receiver anonymity, i.e. topic receiver unlinkablity, The strength of receiver anonymity, i.e. topic receiver unlinkablity,
depends on the number of content topics (`k`), as a proxy for the number of peers and messages, that get mapped onto a single pubsub topic (shard). depends on the number of content topics (`k`), as a proxy for the number of peers and messages, that get mapped onto a single pubsub topic (shard).
For *named* and *static* sharding this responsibility is at the app protocol layer. For _named_ and _static_ sharding this responsibility is at the app protocol layer.
## Copyright ## Copyright
@ -267,12 +272,12 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References ## References
* [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/) - [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/)
* [Unstructured P2P network](https://en.wikipedia.org/wiki/Peer-to-peer#Unstructured_networks) - [Unstructured P2P network](https://en.wikipedia.org/wiki/Peer-to-peer#Unstructured_networks)
* [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) - [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/)
* [WAKU2-ENR](./enr.md) - [WAKU2-ENR](./enr.md)
* [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/) - [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/)
* [Ethereum ENR sharding bit vector](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#metadata) - [Ethereum ENR sharding bit vector](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#metadata)
* [Ethereum discv5 specification](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md) - [Ethereum discv5 specification](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md)
* [Research log: Waku Discovery](https://vac.dev/wakuv2-apd) - [Research log: Waku Discovery](https://vac.dev/wakuv2-apd)
* [WAKU2-RELAY-STATIC-SHARD-ALLOC](../../informational/relay-static-shard-alloc.md) - [WAKU2-RELAY-STATIC-SHARD-ALLOC](../../informational/relay-static-shard-alloc.md)

399
standards/core/store.md
View File

@ -1,22 +1,25 @@
--- ---
slug: 13 title: WAKU2-STORE
title: 13/WAKU2-STORE name: Waku Store Query
name: Waku v2 Store editor: Hanno Cornelius <hanno@status.im>
status: draft
tags: waku-core
editor: Sanaz Taheri <sanaz@status.im>
contributors: contributors:
- Dean Eigenmann <dean@status.im> - Dean Eigenmann <dean@status.im>
- Oskar Thorén <oskarth@titanproxy.com> - Oskar Thorén <oskarth@titanproxy.com>
- Aaryamann Challani <aaryamann@status.im> - Aaryamann Challani <aaryamann@status.im>
- Sanaz Taheri <sanaz@status.im>
---
> **Note:** This version of WAKU2-STORE is earmarked to replace RFC [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/13/store.md) once it reaches draft status
--- ---
# Abstract # Abstract
This specification explains the `13/WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
This specification explains the `WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
stored by other nodes. stored by other nodes.
It also supports pagination for more efficient querying of historical messages. It also supports pagination for more efficient querying of historical messages.
**Protocol identifier***: `/vac/waku/store/2.0.0-beta4` **Protocol identifier***: `/vac/waku/store-query/3.0.0`
## Terminology ## Terminology
The term PII, Personally Identifiable Information, The term PII, Personally Identifiable Information,
@ -28,7 +31,7 @@ the hash of one's static IP address are unique for each user and hence count as
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt). “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol, Nodes willing to provide the storage service using `WAKU2-STORE` protocol,
SHOULD provide a complete and full view of message history. SHOULD provide a complete and full view of message history.
As such, they are required to be *highly available* and As such, they are required to be *highly available* and
specifically have a *high uptime* to consistently receive and store network messages. specifically have a *high uptime* to consistently receive and store network messages.
@ -37,12 +40,12 @@ intact view of the message history is delivered to the querying nodes.
Nevertheless, in case storage provider nodes cannot afford high availability, Nevertheless, in case storage provider nodes cannot afford high availability,
the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past. the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
The concept of `ephemeral` messages introduced in [`14/WAKU2-MESSAGE`](/spec/14) affects `13/WAKU2-STORE` as well. The concept of `ephemeral` messages introduced in [`WAKU2-MESSAGE`](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) affects `WAKU2-STORE` as well.
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](/spec/14). Nodes running `WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md).
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`. Nodes running `WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
# Adversarial Model # Adversarial Model
Any peer running the `13/WAKU2-STORE` protocol, i.e. Any peer running the `WAKU2-STORE` protocol, i.e.
both the querying node and the queried node, are considered as an adversary. both the querying node and the queried node, are considered as an adversary.
Furthermore, Furthermore,
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
@ -51,7 +54,7 @@ As we evolve the protocol,
further adversarial models will be considered. further adversarial models will be considered.
For example, under the passive adversarial model, For example, under the passive adversarial model,
no malicious node hides or no malicious node hides or
lies about the history of messages as it is against the description of the `13/WAKU2-STORE` protocol. lies about the history of messages as it is against the description of the `WAKU2-STORE` protocol.
The following are not considered as part of the adversarial model: The following are not considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections. - An adversary with a global view of all the peers and their connections.
@ -59,139 +62,269 @@ The following are not considered as part of the adversarial model:
In specific, the communication channels are assumed to be secure. In specific, the communication channels are assumed to be secure.
# Wire Specification # Wire Specification
Peers communicate with each other using a request / response API.
The messages sent are Protobuf RPC messages which are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
The following are the specifications of the Protobuf messages.
## Payloads ## Payloads
```protobuf ```protobuf
syntax = "proto3"; syntax = "proto3";
message Index { // Protocol identifier: /vac/waku/store-query/3.0.0
bytes digest = 1; package waku.store.v3;
sint64 receiverTime = 2;
sint64 senderTime = 3; import "waku/message/v1/message.proto";
string pubsubTopic = 4;
message WakuMessageKeyValue {
optional bytes message_hash = 1; // Globally unique key for a Waku Message
// Full message content and associated pubsub_topic as value
optional waku.message.v1.WakuMessage message = 2;
optional string pubsub_topic = 3;
} }
message PagingInfo { message StoreQueryRequest {
uint64 pageSize = 1;
Index cursor = 2;
enum Direction {
BACKWARD = 0;
FORWARD = 1;
}
Direction direction = 3;
}
message ContentFilter {
string contentTopic = 1;
}
message HistoryQuery {
// the first field is reserved for future use
string pubsubtopic = 2;
repeated ContentFilter contentFilters = 3;
PagingInfo pagingInfo = 4;
}
message HistoryResponse {
// the first field is reserved for future use
repeated WakuMessage messages = 2;
PagingInfo pagingInfo = 3;
enum Error {
NONE = 0;
INVALID_CURSOR = 1;
}
Error error = 4;
}
message HistoryRPC {
string request_id = 1; string request_id = 1;
HistoryQuery query = 2; bool include_data = 2; // Response should include full message content
HistoryResponse response = 3;
// Filter criteria for content-filtered queries
optional string pubsub_topic = 10;
repeated string content_topics = 11;
optional sint64 time_start = 12;
optional sint64 time_end = 13;
// List of key criteria for lookup queries
repeated bytes message_hashes = 20; // Message hashes (keys) to lookup
// Pagination info. 50 Reserved
optional bytes pagination_cursor = 51; // Message hash (key) from where to start query (exclusive)
bool pagination_forward = 52;
optional uint64 pagination_limit = 53;
}
message StoreQueryResponse {
string request_id = 1;
optional uint32 status_code = 10;
optional string status_desc = 11;
repeated WakuMessageKeyValue messages = 20;
optional bytes pagination_cursor = 51;
} }
``` ```
## General store query concepts
### Index ### Waku message key-value pairs
To perform pagination, The store query protocol operates as a query protocol for a key-value store of historical Waku messages,
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is associated with a unique `Index` that encapsulates the following parts. with each entry having a [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) and associated pubsub topic as value,
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`. and [deterministic message hash](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md#deterministic-message-hashing) as key.
The hash is computed over the concatenation of `contentTopic` and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](/spec/14)). The store can be queried to return either a set of keys or a set of key-value pairs.
- `receiverTime`: the UNIX time in nanoseconds at which the `WakuMessage` is received by the receiving node. Within the store query protocol, Waku message keys and values MUST be represented in a `WakuMessageKeyValue` message.
- `senderTime`: the UNIX time in nanoseconds at which the `WakuMessage` is generated by its sender. This message MUST contain the deterministic `message_hash` as key.
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received. It MAY contain the full `WakuMessage` and associated pubsub topic as value in the `message` and `pubsub_topic` fields,
depending on the use case as set out below.
If the message contains a value entry in addition to the key,
both the `message` and `pubsub_topic` fields MUST be populated.
The message MUST NOT have either `message` or `pubsub_topic` populated with the other unset.
Both fields MUST either be set or unset.
### PagingInfo ### Waku message store eligibility
`PagingInfo` holds the information required for pagination. It consists of the following components. In order for a Waku message to be eligible for storage:
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery` - it MUST be a _valid_ [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md).
(or retrieved `WakuMessage`s in a `HistoryResponse`). - the `timestamp` field MUST be populated with the Unix epoch time at which the message was generated in nanoseconds.
- `cursor`: holds the `Index` of a `WakuMessage`. If at the time of storage the `timestamp` deviates by more than 20 seconds
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`. either into the past or the future when compared to the store node’s internal clock,
the store node MAY reject the message.
- the `ephemeral` field MUST be set to `false`.
### ContentFilter ### Waku message sorting
`ContentFilter` carries the information required for filtering historical messages.
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](/spec/14).
### HistoryQuery
RPC call to query historical messages. The key-value entries in the store MUST be time-sorted by the `WakuMessage` `timestamp` attribute.
Where two or more key-value entries have identical `timestamps`,
the entries MUST be further sorted by the natural order of their message hash keys.
Within the context of traversing over key-value entries in the store,
_"forward"_ indicates traversing the entries in ascending order,
whereas _"backward"_ indicates traversing the entries in descending order.
- The `pubsubTopic` field MUST indicate the pubsub topic of the historical messages to be retrieved. ### Pagination
This field denotes the pubsub topic on which `WakuMessage`s are published.
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](/spec/11).
Leaving this field empty means no filter on the pubsub topic of message history is requested.
This field SHOULD be left empty in order to retrieve the historical `WakuMessage` regardless of the pubsub topics on which they are published.
- The `contentFilters` field MUST indicate the list of content filters based on which the historical messages are to be retrieved.
Leaving this field empty means no filter on the content topic of message history is required.
This field SHOULD be left empty in order to retrieve historical `WakuMessage` regardless of their content topics.
- `PagingInfo` holds the information required for pagination.
Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`.
It is RECOMMENDED that the queried node defines a maximum page size internally.
If the querying node leaves the `pageSize` unspecified,
or if the `pageSize` exceeds the maximum page size,
the queried node SHOULD auto-paginate the `HistoryResponse` to no more than the configured maximum page size.
This allows mitigation of long response time for `HistoryQuery`.
In the forward pagination request,
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
the `pageSize` amount of `WakuMessage` whose `Index` values are larger than the given `cursor`
(and vise versa for the backward pagination).
Note that the `cursor` of a `HistoryQuery` MAY be empty (e.g., for the initial query), as such, and
depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned, respectively.
### Sorting Messages If a large number of entries in the store service node match the query criteria provided in a `StoreQueryRequest`,
The queried node MUST sort the `WakuMessage` based on their `Index`, the client MAY make use of pagination
where the `senderTime` constitutes the most significant part and the `digest` comes next, and in a chain of store query request and response transactions
then perform pagination on the sorted result. to retrieve the full response in smaller batches termed _"pages"_.
As such, the retrieved page contains an ordered list of `WakuMessage` from the oldest messages to the most recent one. Pagination can be performed either in [a _forward_ or _backward_ direction](#waku-message-sorting).
Alternatively, the `receiverTime` (instead of `senderTime` ) MAY be used to sort messages during the paging process.
However, it is RECOMMENDED the use of the `senderTime` for sorting as it is invariant and
consistent across all the nodes.
This has the benefit of `cursor` reusability i.e.,
a `cursor` obtained from one node can be consistently used to query from another node.
However, this `cursor` reusability does not hold when the `receiverTime` is utilized as the receiver time is affected by the network delay and
nodes' clock asynchrony.
### HistoryResponse A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
by setting the page size limit in the `pagination_limit` field.
Note that a store service node MAY enforce its own limit
if the `pagination_limit` is unset
or larger than the service node's internal page size limit.
RPC call to respond to a HistoryQuery call. A `StoreQueryResponse` with a populated `pagination_cursor` indicates that more stored entries match the query than included in the response.
- The `messages` field MUST contain the messages found,
these are [14/WAKU2-MESSAGE](/spec/14) types. A `StoreQueryResponse` without a populated `pagination_cursor` indicates that
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries. there are no more matching entries in the store.
The `pageSize` indicates the number of returned Waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`).
The `direction` is the same direction as in the corresponding `HistoryQuery`. The client MAY request the next page of entries from the store service node
In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging). by populating a subsequent `StoreQueryRequest` with the `pagination_cursor` received in the `StoreQueryResponse`.
Regardless of the paging direction, the retrieved `messages` are always sorted in ascending order based on their timestamp as explained in the [sorting messages](#sorting-messages) section, that is, from the oldest to the most recent. All other fields and query criteria MUST be the same as in the preceding `StoreQueryRequest`.
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the [14/WAKU2-MESSAGE](/spec/14).
The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result may be different. A `StoreQueryRequest` without a populated `pagination_cursor` indicates that
- The `error` field contains information about any error that has occurred while processing the corresponding `HistoryQuery`. the client wants to retrieve the "first page" of the stored entries matching the query.
`NONE` stands for no error.
This is also the default value. ## Store Query Request
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery` does not match with the `Index` of any of the `WakuMessage` persisted by the queried node.
A client node MUST send all historical message queries within a `StoreQueryRequest` message.
This request MUST contain a `request_id`.
The `request_id` MUST be a uniquely generated string.
If the store query client requires the store service node to include Waku message values in the query response,
it MUST set `include_data` to `true`.
If the store query client requires the store service node to return only message hash keys in the query response,
it SHOULD set `include_data` to `false`.
By default, therefore, the store service node assumes `include_data` to be `false`.
A store query client MAY include query filter criteria in the `StoreQueryRequest`.
There are two types of filter use cases:
1. Content filtered queries and
2. Message hash lookup queries
### Content filtered queries
A store query client MAY request the store service node to filter historical entries by a content filter.
Such a client MAY create a filter on content topic, on time range or on both.
To filter on content topic, the client MUST populate _both_ the `pubsub_topic` _and_ `content_topics` field.
The client MUST NOT populate either `pubsub_topic` or `content_topics` and leave the other unset.
Both fields MUST either be set or unset.
A mixed content topic filter with just one of either `pubsub_topic` or `content_topics` set, SHOULD be regarded as an invalid request.
To filter on time range, the client MUST set `time_start`, `time_end` or both.
Each `time_` field should contain a Unix epoch timestamp in nanoseconds.
An unset `time_start` SHOULD be interpreted as "from the oldest stored entry".
An unset `time_end` SHOULD be interpreted as "up to the youngest stored entry".
If any of the content filter fields are set,
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`,
the client MUST NOT set the `message_hashes` field.
### Message hash lookup queries
A store query client MAY request the store service node to filter historical entries by one or more matching message hash keys.
This type of query acts as a "lookup" against a message hash key or set of keys already known to the client.
In order to perform a lookup query, the store query client MUST populate the `message_hashes` field with the list of message hash keys it wants to lookup in the store service node.
If the `message_hashes` field is set,
the client MUST NOT set any of the content filter fields,
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`.
### Presence queries
A presence query is a special type of lookup query that allows a client to check for the presence of one or more messages in the store service node,
without retrieving the full contents (values) of the messages.
This can, for example, be used as part of a reliability mechanism,
whereby store query clients verify that previously published messages have been successfully stored.
In order to perform a presence query,
the store query client MUST populate the `message_hashes` field in the `StoreQueryRequest` with the list of message hashes
for which it wants to verify presence in the store service node.
The `include_data` property MUST be set to `false`.
The client SHOULD interpret every `message_hash` returned in the `messages` field of the `StoreQueryResponse` as present in the store.
The client SHOULD assume that all other message hashes included in the original `StoreQueryRequest` but not in the `StoreQueryResponse` is not present in the store.
### Pagination info
The store query client MAY include a message hash as `pagination_cursor`,
to indicate at which key-value entry a store service node SHOULD start the query.
The `pagination_cursor` is treated as exclusive
and the corresponding entry will not be included in subsequent store query responses.
For forward queries, only messages following (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor` will be returned.
For backward queries, only messages preceding (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor` will be returned.
If the store query client requires the store service node to perform a forward query,
it MUST set `pagination_forward` to `true`.
If the store query client requires the store service node to perform a backward query,
it SHOULD set `pagination_forward` to `false`.
By default, therefore, the store service node assumes pagination to be backward.
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
by setting the page size limit in the `pagination_limit` field.
Note that a store service node MAY enforce its own limit
if the `pagination_limit` is unset
or larger than the service node's internal page size limit.
See [pagination](#pagination) for more on how the pagination info is used in store transactions.
## Store Query Response
In response to any `StoreQueryRequest`,
a store service node SHOULD respond with a `StoreQueryResponse` with a `requestId` matching that of the request.
This response MUST contain a `status_code` indicating if the request was successful or not.
Successful status codes are in the `2xx` range.
Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
In addition, the store service node MAY choose to provide a more detailed status description in the `status_desc` field.
### Filter matching
For [content filtered queries](#content-filtered-queries), an entry in the store service node matches the filter criteria in a `StoreQueryRequest` if each of the following conditions are met:
- its `content_topic` is in the request `content_topics` set
and it was published on a matching `pubsub_topic` OR the request `content_topics` and `pubsub_topic` fields are unset
- its `timestamp` is _larger or equal_ than the request `start_time` OR the request `start_time` is unset
- its `timestamp` is _smaller_ than the request `end_time` OR the request `end_time` is unset
Note that for content filtered queries, `start_time` is treated as _inclusive_ and `end_time` is treated as _exclusive_.
For [message hash lookup queries](#message-hash-lookup-queries), an entry in the store service node matches the filter criteria if its `message_hash` is in the request `message_hashes` set.
The store service node SHOULD respond with an error code and discard the request
if the store query request contains both content filter criteria and message hashes.
### Populating response messages
The store service node SHOULD populate the `messages` field in the response
only with entries matching the filter criteria provided in the corresponding request.
Regardless of whether the response is to a _forward_ or _backward_ query,
the `messages`field in the response MUST be ordered in a forward direction
according to the [message sorting rules](#waku-message-sorting).
If the corresponding `StoreQueryRequest` has `include_data` set to true,
the service node SHOULD populate both the `message_hash` and `message` for each entry in the response.
In all other cases, the store service node SHOULD populate only the `message_hash` field for each entry in the response.
### Paginating the response
The response SHOULD NOT contain more `messages` than the `pagination_limit` provided in the corresponding `StoreQueryRequest`.
It is RECOMMENDED that the store node defines its own maximum page size internally.
If the `pagination_limit` in the request is unset,
or exceeds this internal maximum page size,
the store service node SHOULD ignore the `pagination_limit` field and apply its own internal maximum page size.
In response to a _forward_ `StoreQueryRequest`:
- if the `pagination_cursor` is set,
the store service node SHOULD populate the `messages` field
with matching entries following the `pagination_cursor` (exclusive).
- if the `pagination_cursor` is unset,
the store service node SHOULD populate the `messages` field
with matching entries from the first entry in the store.
- if there are still more matching entries in the store
after the maximum page size is reached while populating the response,
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
with the message hash key of the _last_ entry _included_ in the response.
In response to a _backward_ `StoreQueryRequest`:
- if the `pagination_cursor` is set,
the store service node SHOULD populate the `messages` field
with matching entries preceding the `pagination_cursor` (exclusive).
- if the `pagination_cursor` is unset,
the store service node SHOULD populate the `messages` field
with matching entries from the last entry in the store.
- if there are still more matching entries in the store
after the maximum page size is reached while populating the response,
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
with the message hash key of the _first_ entry _included_ in the response.
# Security Consideration # Security Consideration
@ -200,11 +333,11 @@ The main security consideration to take into account while using this protocol i
# Future Work # Future Work
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e., - **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
without disclosing the exact topics of [14/WAKU2-MESSAGE](/spec/14) they are interested in. without disclosing the exact topics of [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) they are interested in.
As such, no adversary in the `13/WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e., As such, no adversary in the `WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
content topics of [14/WAKU2-MESSAGE](/spec/14). content topics of [14/WAKU2-MESSAGE](/spec/14).
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity for historical queries, The current version of the `WAKU2-STORE` protocol does not provide anonymity for historical queries,
as the querying node needs to directly connect to another node in the `13/WAKU2-STORE` protocol and as the querying node needs to directly connect to another node in the `WAKU2-STORE` protocol and
explicitly disclose the content filters of its interest to retrieve the corresponding messages. explicitly disclose the content filters of its interest to retrieve the corresponding messages.
However, one can consider preserving anonymity through one of the following ways: However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the request i.e., anonymous communication. - By hiding the source of the request i.e., anonymous communication.
@ -253,8 +386,8 @@ Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/). [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
# References # References
1. [14/WAKU2-MESSAGE](/spec/14) 1. [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md)
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/) 2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
3. [11/WAKU2-RELAY](/spec/11) 3. [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/11/relay.md)
4. [Open timestamps](https://opentimestamps.org/) 4. [Open timestamps](https://opentimestamps.org/)
5. [13/WAKU2-STORE v2 previous version](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/13/store.md)