logos-messaging/specs
1
0
Fork 0
mirror of https://github.com/logos-messaging/specs.git synced 2026-01-02 14:13:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of github.com:waku-org/specs into weboko/reliability-2

Browse Source
This commit is contained in:
Sasha 2024-07-14 02:08:01 +02:00
commit 6436cdc11c
No known key found for this signature in database
19 changed files with 1066 additions and 650 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
.github/workflows/spelling.yml vendored
View File

@ -1,4 +1,3 @@
# This is workflow for spell checking using PySpelling lib (https://pypi.org/project/pyspelling/)
name: Spellcheck
on:
@ -11,12 +10,16 @@ on:
workflow_dispatch:
jobs:
build:
spellcheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: igsekor/pyspelling-any@v1.0.4
name: Spellcheck
env:
MATRIX: ${{ toJson(matrix) }}
run: pyspelling --matrix "$MATRIX"
- name: Install aspell
run: sudo apt-get install -y aspell aspell-en
- name: Run spellcheck
uses: igsekor/pyspelling-any@v1.0.4
with:
args: "-c ./spellcheck.yaml"

16
README.md
View File

@ -26,10 +26,10 @@ This repository contains specifications for the Waku suite of protocols.
|[WAKU2-PEER-EXCHANGE](standards/core/peer-exchange.md)| Waku Peer Exchange |
|[WAKU2-ENR](standards/core/enr.md)| Waku Usage of ENR |
|[WAKU2-INCENTIVIZATION](standards/core/incentivization.md)| Waku Incentivization |
|[WAKU2-METADATA](standards/core/metadata.md)| Waku Metadata |
|[WAKU2-NETWORK](standards/core/network.md)| Waku Network |
|[66/WAKU2-METADATA](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/66/metadata.md)| Waku Metadata |
|[64/WAKU2-NETWORK](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md)| Waku Network |
|[RELAY-SHARDING](standards/core/relay-sharding.md)| Waku Relay Sharding |
| WAKU2-STOREV3 | Coming Soon |
|[WAKU2-STORE](standards/core/store.md) | Waku Store Query |
### Application standards
@ -45,6 +45,8 @@ This repository contains specifications for the Waku suite of protocols.
|[WAKU2-DEVICE-PAIRING](standards/application/device-pairing.md)| Device Pairing and Secure Transfers with Noise |
|[WAKU2-NOISE](standards/application/noise.md)| Noise Protocols for Waku Payload Encryption |
|[TOR-PUSH](standards/application/tor-push.md)| Waku Tor Push |
|[RLN-KEYSTORE](standards/application/rln-keystore.md)| Waku RLN Keystore JSON schema |
|[TRANSPORT-RELIABILITY](standards/application/transport-reliability.md)| Waku Transport Reliability |
### Informational
@ -67,7 +69,13 @@ Relevant Waku resources related to the specifications located in this repository
Contributions are welcome from any party.
Contributors can create specifications relating to the Waku domain and
create a pull request to begin discussion.
The recommended [template](./template.md) may be used for new proposed specifications.
Please adhere to the following contribution guidelines:
- use the recommended [template](./template.md) for new proposed specifications
- use keywords as per the language recommendations in the [template](./template.md) and [Vac COSS](https://github.com/vacp2p/rfc-index/blob/a5b24ac0a27da361312260f9da372a0e6e812212/vac/1/coss.md#language)
- use [semantic breaks](https://sembr.org/)
- links to Waku, Vac or other [IFT](https://free.technology/)-related specifications must be to the corresponding Github repository and not to a webpage.
For example, Waku specs reside in [waku-org/specs](https://github.com/waku-org/specs) and Vac RFCs in [vacp2p/rfc-index](https://github.com/vacp2p/rfc-index/).
New specifications are considered a proof of concept.
Once a rough consensus is reached towards stabilization,

28
informational/adversarial-models.md
View File

@ -13,7 +13,7 @@ This document lists adversarial models and attack-based threats relevant in the
## Motivation and Background
Future versions of this document will serve as a comprehensive list of adversarial models and attack based threats relevant for [Waku v2](https://rfc.vac.dev/spec/10/).
Future versions of this document will serve as a comprehensive list of adversarial models and attack based threats relevant for [Waku v2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md).
The main purpose of this document is being a linkable resource for specifications that address protection as well as mitigation mechanisms within the listed models.
Discussing and introducing countermeasures to specific attacks in specific models is out of scope for this document.
@ -73,7 +73,7 @@ We subdivide anonymity into _receiver anonymity_ and _sender anonymity_.
#### Receiver Anonymity
We define receiver anonymity as _unlinkability of users' identities and the data they receive and/or related actions_.
Because each [Waku message](https://rfc.vac.dev/spec/14/) is associated with a content topic, and each receiver is interested in messages with specific content topics,
Because each [Waku message](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md) is associated with a content topic, and each receiver is interested in messages with specific content topics,
receiver anonymity in the context of Waku corresponds to _subscriber-topic unlinkability_.
An example for the "action" part of our receiver anonymity definition is subscribing to a specific topic.
@ -175,8 +175,8 @@ An entity with this power would, in practice, also have the power of the interna
## Attack-based Threats
The following lists various attacks against [Waku v2](https://rfc.vac.dev/spec/10/) protocols.
If not specifically mentioned, the attacks refer to [Waku relay](https://rfc.vac.dev/spec/11/) and the underlying [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
The following lists various attacks against [Waku v2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md) protocols.
If not specifically mentioned, the attacks refer to [Waku relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) and the underlying [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
We also list the weakest attacker model in which the attack can be successfully performed against.
An attack is considered more powerful if it can be successfully performed in a weaker attacker model.
@ -202,11 +202,11 @@ which in turn significantly increases the probability of attacker nodes ending u
This section lists attacks that aim at deanonymizing a message sender.
We assume that protocol messages are transmitted within a secure channel set up using the [Noise Protocol Framework](https://noiseprotocol.org/).
For [Waku Relay](https://rfc.vac.dev/spec/11/) this means we only consider messages with version field `2`,
For [Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) this means we only consider messages with version field `2`,
which indicates that the payload has to be encoded using [Noise](../standards/application/noise.md).
Note: The currently listed attacks are against libp2p in general.
The [data field of Waku v2 relay](https://rfc.vac.dev/spec/11/#message-fields) must be a [Waku v2 message](https://rfc.vac.dev/spec/14/).
The [data field of Waku v2 relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md/#message-fields) must be a [Waku v2 message](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md).
The attacks listed in the following do not leverage that fact.
#### Replay Attack
@ -217,7 +217,7 @@ Waku relay is inherently safe against replay attack,
because GossipSub nodes, and by extension Waku relay nodes,
feature a `seen` cache, and only relay messages they have not seen before.
Further, replay attacks will be punished by [RLN Relay](https://rfc.vac.dev/spec/17/).
Further, replay attacks will be punished by [RLN Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/17/rln-relay.md).
#### Observing Messages
@ -287,8 +287,8 @@ which can be learned via _graph learning_ attacks.
In a flooding attack, attackers flood the network with bogus messages.
Waku employs [RLN Relay](https://rfc.vac.dev/spec/17/) as the main countermeasure to flooding.
[SWAP](https://rfc.vac.dev/spec/18/) also helps mitigating DoS attacks.
Waku employs [RLN Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/17/rln-relay.md) as the main countermeasure to flooding.
[SWAP](https://github.com/vacp2p/rfc-index/blob/main/waku/deprecated/18/swap.md) also helps mitigating DoS attacks.
#### Black Hole (internal)
@ -317,20 +317,20 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References
- [10/WAKU2](https://rfc.vac.dev/spec/10/)
- [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/)
- [10/WAKU2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md)
- [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [libp2p GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
- [Security](https://en.wikipedia.org/wiki/Information_security)
- [Authentication](https://en.wikipedia.org/wiki/Authentication)
- [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html)
- [Waku v2 message](https://rfc.vac.dev/spec/14/)
- [Waku v2 message](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md)
- [Pluggable Transports](https://www.pluggabletransports.info/about/)
- [Sybil attack](https://en.wikipedia.org/wiki/Sybil_attack)
- [Dolev-Yao model](https://en.wikipedia.org/wiki/Dolev%E2%80%93Yao_model)
- [Noise Protocol Framework](https://noiseprotocol.org/)
- [Noise](../standards/application/noise.md)
- [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/)
- [18/WAKU2-SWAP](https://rfc.vac.dev/spec/18/)
- [17/WAKU-RLN-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/17/rln-relay.md)
- [18/WAKU2-SWAP](https://github.com/vacp2p/rfc-index/blob/main/waku/deprecated/18/swap.md)
- [Dandelion++](https://arxiv.org/abs/1805.11060)
- [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860)
- [Waku Dandelion](../standards/application/dandelion.md))

3
informational/relay-static-shard-alloc.md
View File

@ -26,7 +26,7 @@ this document lists static shard index assignments (see [WAKU2-RELAY-SHARDING](.
| index | Protocol/App | Description |
| ----- | ------------ | --------------------------------------------------------------- |
| 0 | global | global use |
| 1 | reserved | [The Waku Network](https://rfc.vac.dev/spec/64/#network-shards) |
| 1 | reserved | [The Waku Network](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md) |
| 2 | reserved | |
| 3 | reserved | |
| 4 | reserved | |
@ -53,3 +53,4 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
- [WAKU2-RELAY-SHARDING](../standards/core/relay-sharding.md)
- [IANA port allocation](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml)
- [The Waku Network](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md)

36
spellcheck.yaml
View File

@ -2,3 +2,39 @@ matrix:
- name: markdown
sources:
- '**/*.md'
default:
dictionary:
- en_US
- en_GB
encoding: utf-8
pipeline:
- pyspelling.filters.markdown:
extensions:
- .md
- pyspelling.filters.text
sources:
- '**/*.md'
suggest: true
whitelist:
- ALLOC
- IANA
- SHARDING
- WAKU
- Waku
- danielkaiser
- creativecommons
- github
- GITHUB
- https
- iana
- md
- rfc
- RFC
- www
- DHT
- DoS
- GossipSub
- gossipsub
- libp2p
- pubsub
- subnets

80
standards/application/dandelion.md
View File

@ -21,8 +21,8 @@ instead of disseminating messages as per usual relay operation.
## Background and Motivation
[Waku Relay](https://rfc.vac.dev/spec/11/), offers privacy, pseudonymity, and a first layer of anonymity protection by design.
Being a modular protocol family [Waku v2](https://rfc.vac.dev/spec/10/)
[Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md), offers privacy, pseudonymity, and a first layer of anonymity protection by design.
Being a modular protocol family [Waku v2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md)
offers features that inherently carry trade-offs as separate building blocks.
Anonymity protection is such a feature.
The [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html)
@ -30,14 +30,14 @@ states that an anonymous communication network can only have two out of
low bandwidth consumption, low latency, and strong anonymity.
Even when choosing low bandwidth and low latency, which is the trade-off that basic Waku Relay takes,
better anonymity properties (even though not strong per definition) can be achieved by sacrificing some of the efficiency properties.
44/WAKU2-DANDELION specifies one such technique, and aims at gaining the best "bang for the buck"
WAKU2-DANDELION specifies one such technique, and aims at gaining the best "bang for the buck"
in terms of efficiency paid for anonymity gain.
44/WAKU2-DANDELION is based on [Dandelion](https://arxiv.org/abs/1701.04439)
WAKU2-DANDELION is based on [Dandelion](https://arxiv.org/abs/1701.04439)
and [Dandelion++](https://arxiv.org/abs/1805.11060).
Dandelion is a message spreading method, which, compared to other methods,
increases the uncertainty of an attacker when trying to link messages to senders.
Libp2p gossipsub aims at spanning a [d-regular graph](https://en.wikipedia.org/wiki/Regular_graph) topology, with d=6 as the [default value](https://rfc.vac.dev/spec/29/#gossipsub-v10-parameters).
Libp2p gossipsub aims at spanning a [d-regular graph](https://en.wikipedia.org/wiki/Regular_graph) topology, with d=6 as the [default value](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/29/config.md/#gossipsub-v10-parameters).
Messages are forwarded within this (expected) symmetric topology,
which reduces uncertainty when trying to link messages to senders.
Dandelion breaks this symmetry by subdividing message spreading into a "stem" and a "fluff" phase.
@ -61,11 +61,11 @@ Further information on Waku anonymity may be found in our [Waku Privacy and Anon
## Theory and Functioning
44/WAKU2-DANDELION can be seen as an anonymity enhancing add-on to [Waku Relay](https://rfc.vac.dev/spec/11/) message dissemination,
WAKU2-DANDELION can be seen as an anonymity enhancing add-on to [Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) message dissemination,
which is based on [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
44/WAKU2-DANDELION subdivides message dissemination into a "stem" and a "fluff" phase.
WAKU2-DANDELION subdivides message dissemination into a "stem" and a "fluff" phase.
This specification is mainly concerned with specifying the stem phase.
The fluff phase corresponds to [Waku Relay](https://rfc.vac.dev/spec/11/),
The fluff phase corresponds to [Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md),
with optional fluff phase augmentations such as random delays.
Adding random delay in the fluff phase further reduces symmetry in dissemination patterns and
introduces more uncertainty for the attacker.
@ -78,7 +78,7 @@ The Dandelion stem and fluff phases instantiate these concepts.
Future stem specifications might comprise: none (standard relay), Dandelion stem, Tor, and mix-net.
As for future fluff specifications: none (standard relay), diffusion (random delays), and mix-net._
Messages relayed by nodes supporting 44/WAKU2-DANDELION are either in stem phase or in fluff phase.
Messages relayed by nodes supporting WAKU2-DANDELION are either in stem phase or in fluff phase.
We refer to the former as a stem message and to the latter as a fluff message.
A message starts in stem phase, and at some point, transitions to fluff phase.
Nodes, on the other hand, are in stem state or fluff state.
@ -92,14 +92,14 @@ are always sent as stem messages.
The stem phase can be seen as a different protocol, and messages are introduced into Waku Relay, and by extension gossipsub,
once they arrive at a node in fluff state for the first time.
44/WAKU2-DANDELION uses [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) as the protocol for relaying stem messages.
WAKU2-DANDELION uses [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) as the protocol for relaying stem messages.
There are no negative effects on gossipsub peer scoring,
because Dandelion nodes in _stem state_ still normally relay Waku Relay (gossipsub) messages.
## Specification
Nodes $v$ supporting 44/WAKU2-DANDELION MUST either be in stem state or in fluff state.
Nodes $v$ supporting WAKU2-DANDELION MUST either be in stem state or in fluff state.
This does not include relaying messages originated in $v$, for which $v$ SHOULD always be in stem state.
### Choosing the State
@ -114,18 +114,18 @@ corresponding to 10 minute epochs.
### Stem State
On entering stem state,
nodes supporting 44/WAKU2-DANDELION MUST randomly select two nodes for each pubsub topic from the respective gossipsub mesh node set.
nodes supporting WAKU2-DANDELION MUST randomly select two nodes for each pubsub topic from the respective gossipsub mesh node set.
These nodes are referred to as stem relays.
Stem relays MUST support [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/).
If a chosen peer does not support [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/),
Stem relays MUST support [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md).
If a chosen peer does not support [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md),
the node SHOULD switch to fluff state.
(We may update this strategy in future versions of this document.)
Further, the node establishes a map that maps each incoming stem connection
to one of its stem relays chosen at random (but fixed per epoch).
Incoming stem connections are identified by the [Peer IDs](https://docs.libp2p.io/concepts/peers/#peer-id/)
of peers the node receives [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) messages from.
Incoming [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) connections from peers that do not support 44/WAKU2-DANDELION are identified and mapped in the same way.
of peers the node receives [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) messages from.
Incoming [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) connections from peers that do not support WAKU2-DANDELION are identified and mapped in the same way.
This makes the protocol simpler, increases the anonymity set, and offers Dandelion anonymity properties to such peers, too.
The node itself is mapped in the same way, so that all messages originated by the node are relayed via a per-epoch-fixed Dandelion relay, too.
@ -133,7 +133,7 @@ The node itself is mapped in the same way, so that all messages originated by th
While in stem state, nodes MUST relay stem messages to the respective stem relay.
Received fluff messages MUST be relayed as specified in the fluff state section.
The stem protocol ([19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/)) is independent of the fluff protocol ([Waku Relay](https://rfc.vac.dev/spec/11/)).
The stem protocol ([19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md)) is independent of the fluff protocol ([Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)).
While in stem state, nodes MUST NOT gossip about stem messages,
and MUST NOT send control messages related to stem messages.
(An existing gossipsub implementation does _not_ have to be adjusted to not send gossip about stem messages,
@ -161,7 +161,7 @@ this regular forwarding already comprises random delays.
## Implementation Notes
Handling of the 44/WAKU2-DANDELION stem phase can be implemented as an extension to an existing [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) implementation.
Handling of the WAKU2-DANDELION stem phase can be implemented as an extension to an existing [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) implementation.
Fluff phase augmentations might alter gossipsub message dissemination (e.g. adding random delays).
If this is the case, they have to be implemented on the libp2p gossipsub layer.
@ -182,22 +182,22 @@ The fail-safe mechanism specified in this document (proposed in the Dandelion pa
#### Attacker Model and Anonymity Goals
44/WAKU2-DANDELION provides significant mitigation against mass deanonymization in the
WAKU2-DANDELION provides significant mitigation against mass deanonymization in the
passive [scaling multi node model](../../informational/adversarial-models.md/#scaling-multi-node).
in which the attacker controls a certain percentage of nodes in the network.
44/WAKU2-DANDELION provides significant mitigation against mass deanonymization
WAKU2-DANDELION provides significant mitigation against mass deanonymization
even if the attacker knows the network topology, i.e. the anonymity graph and the relay mesh graph.
Mitigation in stronger models, including the _active scaling multi-node_ model, is weak.
We will elaborate on this in future versions of this document.
44/WAKU2-DANDELION does not protect against targeted deanonymization attacks.
WAKU2-DANDELION does not protect against targeted deanonymization attacks.
#### Non-Dandelion Peers
Stem relays receiving messages can either be in stem state or in fluff state themselves.
They might also not support 44/WAKU2-DANDELION,
and interpret the message as classical [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/),
They might also not support WAKU2-DANDELION,
and interpret the message as classical [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md),
which effectively makes them act as fluff state relays.
While such peers lower the overall anonymity properties,
the [Dandelion++ paper](https://arxiv.org/abs/1805.11060)
@ -218,7 +218,7 @@ Generally, there are several design choices to be made for the stem phase of a D
#### Bound Stem Length
Choosing $q = 0.2$, 44/WAKU2-DANDELION has an expected stem length of 5 hops,
Choosing $q = 0.2$, WAKU2-DANDELION has an expected stem length of 5 hops,
Assuming $100ms$ added delay per hop, the stem phase adds around 500ms delay on average.
There is a possibility for the stem to grow longer,
@ -233,10 +233,10 @@ We will quantify the resulting loss of anonymity in future versions of this docu
#### Stem Relay Selection
In its current version, 44/WAKU2-DANDELION nodes default to fluff state
if the random stem relay selection yields at least one peer that does not support [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) (which is the stem protocol used in [44/WAKU2-DANDELION](https://rfc.vac.dev/spec/44/).
If nodes would reselect peers until they find peers supporting [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/),
malicious nodes would get an advantage if a significant number of honest nodes would not support [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19).
In its current version, WAKU2-DANDELION nodes default to fluff state
if the random stem relay selection yields at least one peer that does not support [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) (which is the stem protocol used in WAKU2-DANDELION.
If nodes would reselect peers until they find peers supporting [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md),
malicious nodes would get an advantage if a significant number of honest nodes would not support [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md).
Even though this causes messages to enter fluff phase earlier,
we choose the trade-off in favour of protocol stability and sacrifice a bit of anonymity.
(We will look into improving this in future versions of this document.)
@ -245,7 +245,7 @@ we choose the trade-off in favour of protocol stability and sacrifice a bit of a
[Dandelion](https://arxiv.org/abs/1701.04439) and [Dandelion++](https://arxiv.org/abs/1805.11060)
assume adding random delays in the fluff phase as they build on Bitcoin diffusion.
44/WAKU2-DANDELION (in its current state) allows for zero delay in the fluff phase and outsources fluff augmentations to dedicated specifications.
WAKU2-DANDELION (in its current state) allows for zero delay in the fluff phase and outsources fluff augmentations to dedicated specifications.
While this lowers anonymity properties, it allows making Dandelion an opt-in solution in a given network.
Nodes that do not want to use Dandelion do not experience any latency increase.
We will quantify and analyse this in future versions of this specification.
@ -254,7 +254,7 @@ We plan to add a separate fluff augmentation specification that will introduce r
Optimal delay times depend on the message frequency and patterns.
This delay fluff augmentation specification will be oblivious to the actual message content,
because Waku Dandelion specifications add anonymity on the routing layer.
Still, it is important to note that [Waku2 messages](https://rfc.vac.dev/spec/14/#payloads) (in their current version) carry an originator timestamp,
Still, it is important to note that [Waku2 messages](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md/#payloads) (in their current version) carry an originator timestamp,
which works against fluff phase random delays.
An analysis of the benefits of this timestamp versus anonymity risks is on our roadmap.
@ -266,15 +266,15 @@ Note: Introducing random delays can have a negative effect on
#### Stem Flag
While 44/WAKU2-DANDELION without fluff augmentation does not effect Waku Relay nodes,
messages sent by nodes that only support [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) might be routed through a Dandelion stem without them knowing.
While WAKU2-DANDELION without fluff augmentation does not effect Waku Relay nodes,
messages sent by nodes that only support [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) might be routed through a Dandelion stem without them knowing.
While this improves anonymity, as discussed above, it also introduces additional latency and lightpush nodes cannot opt out of this.
In future versions of this specification we might
- add a flag to [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) indicating a message should be routed over a Dandelion stem (opt-in), or
- add a flag to [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) indicating a message should _not_ be routed over a Dandelion stem (opt-out), or
- introducing a fork of [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) exclusively used for Dandelion stem.
- add a flag to [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md) indicating a message should be routed over a Dandelion stem (opt-in), or
- add a flag to [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md) indicating a message should _not_ be routed over a Dandelion stem (opt-out), or
- introducing a fork of [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) exclusively used for Dandelion stem.
In the current version, we decided against these options in favour of a simpler protocol and an increased anonymity set.
@ -287,11 +287,13 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
- [Dandelion](https://arxiv.org/abs/1701.04439)
- [Dandelion++](https://arxiv.org/abs/1805.11060)
- [multi-node (botnet) attacker model](../../informational/adversarial-models.md/#multi-node)
- [Waku Relay](https://rfc.vac.dev/spec/11/)
- [Waku v2](https://rfc.vac.dev/spec/10/)
- [Waku Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [Waku v2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md)
- [d-regular graph](https://en.wikipedia.org/wiki/Regular_graph)
- [Anonymity Trilemma](https://freedom.cs.purdue.edu/projects/trilemma.html)
- [Waku Privacy and Anonymity Analysis](https://vac.dev/wakuv2-relay-anon).
- [On the Anonymity of Peer-To-Peer Network Anonymity Schemes Used by Cryptocurrencies](https://arxiv.org/pdf/2201.11860.pdf)
- [Adversarial Models](https://rfc.vac.dev/spec/45/)
- [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/)
- [Adversarial Models](../../informational/adversarial-models.md)
- [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md)
- [29/WAKU-CONFIG](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/29/config.md)
- [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md)

5
standards/application/device-pairing.md
View File

@ -55,7 +55,7 @@ This protocol is designed in order to achieve two main security objectives:
The devices execute a custom handshake derived from `XX`,
where they mutually exchange and authenticate their respective device static key
by exchanging messages over the content topic with the following [format](https://rfc.vac.dev/spec/23/#content-topic-format)
by exchanging messages over the content topic with the following [format](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/23/topics.md/#content-topic-format)
```
contentTopic = /{application-name}/{application-version}/wakunoise/1/sessions_shard-{shard-id}/proto
@ -369,7 +369,8 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
### Informative
- [26/WAKU2-PAYLOAD](https://rfc.vac.dev/spec/35/#abnf)
- [26/WAKU2-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md/#abnf)
- [format](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/23/topics.md/#content-topic-format)
- [The Double-Ratchet Algorithm](https://signal.org/docs/specifications/doubleratchet/)
- [The Noise Protocol Framework specifications](http://www.noiseprotocol.org/noise.html)
- [IETF RFC 4648 - The Base16, Base32, and Base64 Data Encodings](https://datatracker.ietf.org/doc/html/rfc4648)

4
standards/application/noise-sessions.md
View File

@ -60,7 +60,7 @@ The above mechanism allows a Noise session to be marked as stale either privatel
depending if `Hash(session-id)` is sent on `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto` to the other party in encrypted form or not, respectively.
When a Noise session is publicly marked as stale,
network peers MAY discard all [stored](https://rfc.vac.dev/spec/13/) messages addressed to the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`.
network peers MAY discard all [stored](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md) messages addressed to the content topic `/{application-name}/{application-version}/wakunoise/1/sessions/{ct-id}/proto`.
In this the case and in order for parties to retrieve any eventually delayed message,
peers SHOULD wait a fixed amount of time before discarding stored messages corresponding to a stale Noise session.
@ -150,7 +150,7 @@ This session management mechanism is loosely based on [Signal's Sesame Algorithm
# References
- [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/)
- [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md)
- [WAKU2-NOISE](./noise.md)
- [The Noise Protocol Framework](http://www.noiseprotocol.org/noise.html)
- [The Sesame Algorithm: Session Management for Asynchronous Message Encryption](https://signal.org/docs/specifications/sesame/)

24
standards/application/noise.md
View File

@ -6,11 +6,11 @@ editor: Giuseppe <giuseppe@status.im>
contributors:
---
This specification describes how payloads of [Waku messages](https://rfc.vac.dev/spec/14/) with [version 2](https://rfc.vac.dev/spec/14/#version2) can be encrypted
This specification describes how payloads of [Waku messages](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14) with [version 2](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14) can be encrypted
in order to achieve confidentiality, authenticity, and integrity
as well as some form of identity-hiding on communicating parties.
This specification extends the functionalities provided by [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/),
This specification extends the functionalities provided by [26/WAKU-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md),
adding support to modern symmetric encryption primitives
and asymmetric key-exchange protocols.
@ -58,7 +58,7 @@ We note that all [design requirements](#Design-requirements) on exchanged messag
corresponding to a total of 1 Round Trip Time communication _(1-RTT)_.
In particular, identity-hiding properties can be guaranteed only if the recommendation described in [After-handshake](#After-handshake) are implemented.
In the following, we assume that communicating parties reciprocally know an initial [`contentTopic`](https://rfc.vac.dev/spec/14/#wakumessage)
In the following, we assume that communicating parties reciprocally know an initial [`contentTopic`](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14/#wakumessage)
where they can send/receive the first handshake message(s).
We further assume that messages sent over a certain `contentTopic` can be efficiently identified by their intended recipients
thanks to an arbitrary 16 bytes long `message-nametag` field embedded in the message payload
@ -82,7 +82,7 @@ The symmetric primitives supported are:
## Specification
When [14/WAKU-MESSAGE version](https://rfc.vac.dev/spec/14/#payload-encryption) is set to 2,
When [14/WAKU-MESSAGE version](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14/#payload-encryption) is set to 2,
the corresponding `WakuMessage`'s `payload` will encapsulate the two fields `handshake-message` and `transport-message`.
The `handshake-message` field MAY contain
@ -218,26 +218,26 @@ by hashing the result of an ephemeral-ephemeral Diffie-Hellman exchange every 1-
## Backward Support for Symmetric/Asymmetric Encryption
It is possible to have backward compatibility to symmetric/asymmetric encryption primitives from [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/),
effectively encapsulating payload encryption [14/WAKU-MESSAGE version 1](https://rfc.vac.dev/spec/14/#version1) in [version 2](https://rfc.vac.dev/spec/14/#version2).
It is possible to have backward compatibility to symmetric/asymmetric encryption primitives from [26/WAKU-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md),
effectively encapsulating payload encryption [14/WAKU-MESSAGE version 1](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14/#version1) in [version 2](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14/#version2).
It suffices to extend the list of supported `protocol-id` to:
- `254`: AES-256-GCM symmetric encryption;
- `255`: ECIES asymmetric encryption.
and set the `transport-message` field to the [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26) `data` field, whenever these `protocol-id` values are set.
and set the `transport-message` field to the [26/WAKU-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md) `data` field, whenever these `protocol-id` values are set.
Namely, if `protocol-id = 254, 255` then:
- `message-nametag`: is empty;
- `handshake-message-len`: is set to `0`;
- `handshake-message`: is empty;
- `transport-message`: contains the [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/) `data` field (AES-256-GCM or ECIES, depending on `protocol-id`);
- `transport-message`: contains the [26/WAKU-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md) `data` field (AES-256-GCM or ECIES, depending on `protocol-id`);
- `transport-message-len` is set accordingly to `transport-message` length;
When a `transport-message` corresponding to `protocol-id = 254, 255` is retrieved,
it SHOULD be decoded as the `data` field in [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26/) specification.
it SHOULD be decoded as the `data` field in [26/WAKU-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md) specification.
## Appendix: Supported Handshakes Description
@ -318,9 +318,9 @@ The main difference with `XX` is that Alice's and Bob's static keys, when transm
## References
1. [5/SECURE-TRANSPORT](https://specs.status.im/spec/5)
2. [10/WAKU2](https://rfc.vac.dev/spec/10)
3. [26/WAKU-PAYLOAD](https://rfc.vac.dev/spec/26)
4. [14/WAKU-MESSAGE](https://rfc.vac.dev/spec/14/#version1)
2. [10/WAKU2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md)
3. [26/WAKU-PAYLOAD](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/application/26/payload.md)
4. [14/WAKU-MESSAGE](https://github.com/vacp2p/rfc-index/tree/main/waku/standards/core/14)
5. [Noise protocol](http://www.noiseprotocol.org/noise.html)
6. [Noise handshakes as key-exchange mechanism for Waku2](https://forum.vac.dev/t/noise-handshakes-as-key-exchange-mechanism-for-waku2/130)
7. [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234)

299
standards/application/rln-keystore.md Normal file
View File

@ -0,0 +1,299 @@
---
title: WAKU-RLN-KEYSTORE
name: Waku RLN Keystore
category: Standards Track
editor: Jimmy Debe <jimmy@status.im>
contributors:
- Aaryamann Challani <aaryamann@status.im>
---
## Abstract
This specification describes how RLN, Rate Limit Nullifier,
credentials are securely stored in a JSON schema.
## Summary
A keystore is a construct to store a user’s keys.
The keys will be encrypted and decrypted based on methods specified in this specification.
The keystore stores a user's credentials locally and
uses [32/RLN-V1](/spec/32/) as a spam-prevention mechanism by generating zero-knowledge proofs.
## Background
The secure storage of keys is important in peer-to-peer messaging applications.
A `WAKU-RLN-KEYSTORE` uses zero-knowledge proofs for anonymous rate-limiting for messaging frameworks.
Generated credentials by a user are encrypted and stored in the keystore to be retrieved over a network.
With [32/RLN-V1](/spec/32/), sending and receiving
messages will ensure a message rate for a network is being followed while preserving the anonymity of the message owner.
### Waku RLN Keystore Format:
A format example of a keystore used by a [17/WAKU2-RLN-Relay](/spec/17/).
```js
const Keystore {
application: "waku-rln-relay" ,
appIdentifier: "string",
version: "string",
credentials: {
"membershipHash": {
crypto: {
cipher: "string",
cipherparams: {
iv: "string",
},
ciphertext: "string",
kdf: "string",
kdfparams: {
dklen: integer,
c: integer,
prf: "string",
salt: "string",
},
mac: "string",
}
}
}
}
```
## Specification
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”,
“NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
The keystore MUST be generated by a cryptographic construction with password verification and decryption.
Keystore modules MUST include metadata, key derivation function, checksum, cipher, and a membership hash.
### Metadata:
Information about the keystore SHOULD be stored in the metadata.
The declaration of `application`, `version`, and `appIdentifier` COULD occur in the metadata.
- `application` : current application, MUST be a string
- `version` : application version, MUST be a string, SHOULD follow semantic versioning
- `appIdentifier`: application identifier, MUST be a string
### Credentials:
After RLN credentials are generated, it MUST be stored in a JSON schema.
The Waku RLN credentials MUST consist of a `membershipHash` and `WakuCredential`.
The `membershipHash` will be an identity hash of the user.
The `WakuCredential` will store to encryption portion of the keystore.
There COULD be multiple credentials stored in a keystore, categorized by the `membershipHash`.
Each contruct MUST include the keypair:
> key: [`membershipHash`]: pair: [`WakuCredential`]
#### membershipHash
The `membershipHash` SHOULD be generated by user's participating in a membership group,
as decribed in [32/RLN-V1](/spec/32/).
Each user SHOULD register to the group with an `identity_commitment` stored in a Merkle tree.
A cryptographic hash function that SHOULD be used to generate the `membershipHash` is [SHA256](https://www.rfc-editor.org/rfc/rfc4634.txt),
other hash functions MAY be used.
The hash function that is used,
SHOULD be mentioned in the `verison` attribute.
To generate the `membershipHash`,
the `treeIndex`, `membershipContract`, `chainId` and `identityCredential` attributes SHOULD be used to create a hexadecimal string.
- it MUST NOT already exist in the keystore.
##### `treeIndex`
After a user registers to a group,
a `treeIndex` value of the position in the Merkle tree SHOULD be returned.
- it MUST be a Merkle tree data structure filled with `identity_commitment` from user registrations.
- it MUST be a hexadecimal string
##### `membershipContract`
For decentralized membership registrations,
the `membershipContract` SHOULD be a `contractAddress` from a public blockchain using smart contracts.
- it MUST be a string.
##### `chainId`
It uniquely defines the chain upon which the registration has occurred.
The `chainId` SHOULD be the blockchain identifier used for `membershipcontract`,
as described in [EIP155](https://eips.ethereum.org/EIPS/eip-155).
- it MUST be a string
##### `identityCredential`
The `identityCredential` MUST be derived after a succussful decryption of the keystore.
The `identityCredential` MUST be constructed with the `identity_secret`, `identity_secret_hash`, `identity_commitment` values.
- it MUST be a hash of `identity_commitment` stored in a Merkle tree.
- it MUST be a string.
###### `identity_secret`
The `identity_secret` MUST be constructed with `identity_nullifier` + `identity_trapdoor` values.
- `identity_nullifier` : Random 32 byte value
- `identity_trapdoor` : Random 32 byte value
###### `identity_secret_hash`
Used to derive the `identity_commitment` of the user, and
as a private input for zero-knowledge proof generation.
- it MUST be created with `identity_secret` as a parameter for the hash function.
- This secret hash SHOULD be kept private by the user.
###### `identity_commitment`
- it SHOULD be created with `identity_secret_hash` by using the hash function Poseidon,
as described in [Poseidon Paper](https://eprint.iacr.org/2019/458.pdf).
- it MUST be used by a user for group registering.
#### WakuCredential
The `WakuCredential` will store values used for encrypting and decrypting user's keystores.
- it MUST be used for password verification.
- it MUST follow [EIP-2335](https://eips.ethereum.org/EIPS/eip-2335)
- it SHOULD use [SHA256](https://www.rfc-editor.org/rfc/rfc4634.txt) as the hash function
#### KDF
The password-based encryption used SHOULD be KDF, key derivation function,
to produce a derived key from a password and other parameters.
The keystore COULD use PBKDF2 password-based encryption,
as described in [RFC 2898](https://www.ietf.org/rfc/rfc2898.txt).
A `WakuCredential` object MUST include:
| Name | Description |
|----|-----|
| password | used to encrypt keystore and decryption key |
| secret | key to be encrypted |
| pubKey | public key |
| path | HD, hardened derivation, path used to generate the secret |
| checksum | hash function |
| cipher | cipher function |
```js
crypto: {
cipher: "string" // The cipher function
cipherparams: {
iv: "string" // The cipher parameters
},
ciphertext: "string" // The cipher message,
kdf: "string" // KDF Function,
kdfparams: {
param: integer // Salt value and iteration count,
dklen: integer // Length in octets of derived key, MUST be positive integer,
c: "string" // Iteration count, MUST be positive integer,
prf: "string" // Underlying pseudorandom function,
salt: "string" // Produces a large set of keys based on the password
},
mac: "string" // Checksum
}
```
#### Decryption
The keystore SHOULD decrypt a user's credentials using a password and the `membershipHash`,
using PBKDF2 that returns the `decryptionKey` key.
The decryption key is used to verify the keystore is correct.
- To generate the `decryptionKey`, it MUST be constructed from a password and KDF,
as desrcibed in [ERC-2335: BLS12-381 Keystore](https://eips.ethereum.org/EIPS/eip-2335).
- The `decryptionKey`, is derived from the cipher function and
cipher parameters described in the KDF used in the keystore.
### Test Vectors
RLN uses Poseidon hash algorithm to generate the `identityCredential`,
as described in [Poseidon Paper](https://eprint.iacr.org/2019/458.pdf).
The keystore hash algorithm used is [SHA256](https://www.rfc-editor.org/rfc/rfc4634.txt).
#### Input:
- `application`: "waku-rln-relay"
- `appIdentifier`: "01234567890abcdef"
- `version`: "0.2"
- `hashFunction`: "poseidonHash"
- `password`: "sup3rsecure"
```js
identityCredential = {
IDTrapdoor: [
211, 23, 66, 42, 179, 130, 131, 111, 201, 205, 244, 34, 27, 238, 244,
216, 131, 240, 188, 45, 193, 172, 4, 168, 225, 225, 43, 197, 114, 176,
126, 9,
],
IDNullifier: [
238, 168, 239, 65, 73, 63, 105, 19, 132, 62, 213, 205, 191, 255, 209, 9,
178, 155, 239, 201, 131, 125, 233, 136, 246, 217, 9, 237, 55, 89, 81,
42,
],
IDSecretHash: [
150, 54, 194, 28, 18, 216, 138, 253, 95, 139, 120, 109, 98, 129, 146,
101, 41, 194, 36, 36, 96, 152, 152, 89, 151, 160, 118, 15, 222, 124,
187, 4,
],
IDCommitment: [
112, 216, 27, 89, 188, 135, 203, 19, 168, 211, 117, 13, 231, 135, 229,
58, 94, 20, 246, 8, 33, 65, 238, 37, 112, 97, 65, 241, 255, 93, 171, 15,
],
}
membership = {
chainId: "0xAA36A7",
treeIndex: 8,
address: "0x8e1F3742B987d8BA376c0CBbD7357fE1F003ED71",
}
```
#### Output:
```js
application: "waku-rln-relay",
appIdentifier: "01234567890abcdef",
version: "0.2",
credentials: {
"9DB2B4718A97485B9F70F68D1CC19F4E10F0B4CE943418838E94956CB8E57548": {
crypto: {
cipher: "aes-128-ctr",
cipherparams: {
iv: "fd6b39eb71d44c59f6bf5ff3d8945c80",
},
ciphertext: "9c72f47ce95de03ed34502d0288e7576b66b51b9e7d5ae882c27bd89f94e6a03c2c44c2ddf0c982e72003d67212105f1b64614f57cabb0ceadab7e07be165eee1121ad6b81951368a9f3be2dd99ea294515f6013d5f2bd4702a40e36cfde2ea298b23b31e5ce719d8040c3331f73d6bf44f88bca39bac0e917d8bf545500e4f40d321c235426a80f315ac70666acbd3bdf803fbc1e7e7103fed466525ed332b25d72b2dbedf6fa383b2305987c1fe276b029570519b3e79930edf08c1029868d05c2c08ab61d7c64f63c054b4f6a5a12d43cdc79751b6fe58d3ed26b69443eb7c9f7efce27912340129c91b6b813ac94efd5776a40b1dda896d61357de208c7c47a14af911cc231355c8093ee6626e89c07e1037f9e0b22c690e3e049014399ca0212c509cb04c71c7860d1b17a0c47711c490c27bad2825926148a1f15a507f36ba2cdaa04897fce2914e53caed0beaf1bebd2a83af76511cc15bff2165ff0860ad6eca1f30022d7739b2a6b6a72f2feeef0f5941183cda015b4631469e1f4cf27003cab9a90920301cb30d95e4554686922dc5a05c13dfb575cdf113c700d607896011970e6ee7d6edb61210ab28ac8f0c84c606c097e3e300f0a5f5341edfd15432bef6225a498726b62a98283829ad51023b2987f30686cfb4ea3951f3957654035ec291f9b0964a3a8665d81b16cec20fb40f944d5f9bf03ac1e444ad45bae3fa85e7465ce620c0966d8148d6e2856f676c4fbbe3ebe470453efb4bbda1866680037917e37765f680e3da96ef3991f3fe5cda80c523996c2234758bf5f7b6d052dc6942f5a92c8b8eec5d2d8940203bbb6b1cba7b7ebc1334334ca69cdb509a5ea58ec6b2ebaea52307589eaae9430eb15ad234c0c39c83accdf3b77e52a616e345209c5bc9b442f9f0fa96836d9342f983a7",
kdf: "pbkdf2",
kdfparams: {
dklen: 32,
c: 1000000,
prf: "hmac-sha256",
salt: "60f0aa92fbf63a8356dfdbed2ab18058",
},
mac: "51a227ac6db7f2797c63925880b3db664e034231a4c68daa919ab42d8df38bc6",
},
}
```
## Security Considerations
### 1.) Add a Password
An attacker can identify which credential belongs to a combination of `chainId` and
`contractAddress` pair by brute forcing the `treeIndex` iteratively to find a hash match.
The RECOMMENDED solution is to add a password to the construction of `membershipHash` to prevent this attack.
The RECOMMENDED `membershipHash` Construction:
- `membershipHash` SHOULD be constructed with `treeIndex`, `membershipContract`, `identityCredential`, `membershipPassword`
- `membershipPassword` : a new password created to private attacks compromising keystore credentials.
- The user MUST store the `membershipPassword` privately.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [32/RLN-V1](/spec/32/)
2. [17/WAKU2-RLN-RELAY](/spec/17/)
3. [SHA256](https://www.rfc-editor.org/rfc/rfc4634.txt)
4. [EIP155](https://eips.ethereum.org/EIPS/eip-155)
5. [Poseidon Paper](https://eprint.iacr.org/2019/458.pdf)
6. [EIP-2335](https://eips.ethereum.org/EIPS/eip-2335)
7. [RFC 2898](https://www.ietf.org/rfc/rfc2898.txt)
8. [ERC-2335: BLS12-381 Keystore](https://eips.ethereum.org/EIPS/eip-2335)

14
standards/application/tor-push.md
View File

@ -9,10 +9,10 @@ contributors:
## Abstract
This document extends the [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11), specifying Waku Tor Push,
This document extends the [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md), specifying Waku Tor Push,
which allows nodes to push messages via Tor into the Waku relay network.
Waku Tor Push builds on [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46).
Waku Tor Push builds on [GOSSIPSUB-TOR-PUSH](https://github.com/vacp2p/rfc-index/blob/main/vac/raw/gossipsub-tor-push.md).
**Protocol identifier**: /vac/waku/relay/2.0.0
@ -22,13 +22,13 @@ This allows Waku relay nodes that are oblivious to Tor Push to process messages
## Functional Operation
In its current version, Waku Tor Push corresponds to [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46)
applied to [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/),
In its current version, Waku Tor Push corresponds to [GOSSIPSUB-TOR-PUSH](https://github.com/vacp2p/rfc-index/blob/main/vac/raw/gossipsub-tor-push.md)
applied to [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md),
instead of [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md).
## Security/Privacy Considerations
see [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46)
see [GOSSIPSUB-TOR-PUSH](https://github.com/vacp2p/rfc-index/blob/main/vac/raw/gossipsub-tor-push.md)
## Copyright
@ -36,7 +36,7 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References
- [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11)
- [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [libp2p gossipsub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md)
- [46/GOSSIPSUB-TOR-PUSH](https://rfc.vac.dev/spec/46)
- [GOSSIPSUB-TOR-PUSH](https://github.com/vacp2p/rfc-index/blob/main/vac/raw/gossipsub-tor-push.md)
- [Tor](https://www.torproject.org/)

196
standards/application/transport-reliability.md Normal file
View File

@ -0,0 +1,196 @@
---
title: TRANSPORT-RELIABILITY
name: Waku Transport Reliability
category: Standards Track
tags: [reliability, application]
editor: Kaichao Sun <kaichao@status.im>
contributors:
- Richard Ramos <richard@status.im>
---
## Abstract
Waku provides an efficient transport layer for p2p communications.
It defines protocols like [Relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) and [Lightpush](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md) / [Filter](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/12/filter.md) for routing messages in decentralised networks.
However, there is no guarantee that a message broadcast in a Waku network will reach its destination.
For example, the receiver in a chat application using Waku as p2p transport may miss messages when a network issue happens at either the sender or the receiver side.
In general, a message in a Waku network may be in one of 3 states from the sender's perspective:
- **outgoing**, the message is posted by the sender but no confirmations from other nodes yet
- **sent**, the message is received by any other node in the network
- **delivered**, the message is acknowledged on the application layer by the intended recipient
Application like Status already uses [MVDS](https://github.com/vacp2p/rfc-index/blob/main/vac/2/mvds.md) for e2e acknowledgement in direct messages and group chat. Also there is an ongoing [discussion](https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293) about a more general and bandwidth efficient solution for e2e reliablity.
In other words, an application defines a payload over Waku and is interested in e2e delivery between application users. Waku provides a pub/sub broadcast transport, which is interested in reliably routing a message to all participants in the broadcast group.
Before we have a complete design for e2e reliability, we need to compose existing protocols to increase the reliability of the transport protocol. This document proposes a few options for such composition.
## Motivation
The [Store protocol](https://github.com/waku-org/specs/blob/master/standards/core/store.md) provides a way for nodes in the network to query the existence of messages or fetch specific messages based on the search criteria.
**Query criteria with message hash**
For the nodes that may have connection issues to **publish** messages via transport layer, this search criteria can be used to check whether a message is populated in the network or not. The message exists in store node can be marked from `outgoing` to `sent` by application. If the message is not found in the store node, the application can resend the message.
**Search criteria with topics and time range**
For the nodes that may have connection issues to **receive** messages via transport layer, this search criteria can be used to fetch missing messages from store nodes periodically after network resumes.
In summary, by leveraging the store node to provide such query services, the applications are able to mitigate reliability issues on the Waku transport layer.
This approach also introduces new limitations like centralized points of failures in Store nodes, diminished privacy, and lower scalability.
It should be viewed as a temporary solution and deprecated when an e2e reliability solution is ready.
## Implementation Suggestions
### Query with Message Hash
For outgoing messages, the processing flow can be like this:
- create a buffer for all "outgoing" message hashes
- send message via relay or lightpush protocol
- add message hash to the buffer
- keep a copy of the message locally with status outgoing
- check the buffer periodically
- query the store node with message hash in the buffer of which the send attempt was more than a few seconds ago
- if the message exists, update its status to "sent" in local data store and remove the message hash from the buffer
- if the message does not exist, resend the message
- if the message is still missing in the store node for a period of time, trigger the message failed to send workflow and remove the message hash from the buffer
The implementation in Python may look like this:
```python
outgoingMessageHashes = []
class Message:
hash: str
postTime: int
status: str
content: str
def send(message):
# send message via relay or lightpush protocol, here use relay as example
waku.relay.post(message)
outgoingMessageHashes.append(message.hash)
message.status = 'outgoing'
database.saveMessage(message)
def checkOutgoingMessages(peerID):
for messageHash in outgoingMessageHashes:
message = database.getMessage(messageHash)
# only query store node for ongoing message, and posted more than 3 seconds ago
if message.status == 'ongoing' && time.now() - message.postTime > 3:
response = waku.store.queryMessage(peerID, messageHash)
if response.exists():
database.updateMessageStatus(messageHash, 'sent')
outgoingMessageHashes.remove(messageHash)
elif time.now() - message.postTime > 10:
# resend the message if it's not stored in store node after 10 seconds
waku.relay.post(message)
```
Function `checkOutgoingMessages` is called periodically, most likely every a few seconds. Message hashes can be queried in batch to reduce the number of requests to store nodes, the size in a batch shoud not exceed the max supported size by store node.
The store node can be set and updated directly by application or selected from peers which are discovery by protocols like [discv5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md) or [peer exchange](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md).
The store node may only support specific pubsub topics, and the application should group message hashes by pubsub topic before sending the request.
When persistent network issue happens, you may not want to resend the failed messages indefinitely, the application should have a mechanism to clean the cache with failed message hashes and trigger other retry logic after a few attempts.
### Query with Topics and Time Range
An application could use different pubsub topics and content topics, for example a community may have its own pubsub topic, and each channel may have its own content topic. To fetch all missing messages in a specific channel, the application can query the store node with the provided pubsub topic, content topic and time range.
For incoming messages, the processing flow can be like this:
- subscribe to the interested pubsub and content topics
- periodically query the store node with the interested topics and time range for message hashes
- check if each received message hash already exists in the local database. if not, add the missing message hash to a buffer.
- batch fetch the full messages corresponding to the missing message hashes in the buffer from the store node
- process the messages
The implementation in Python may look like this:
```python
class FetchRecord:
pubsubTopic: str
lastFetch: int
class QueryParams:
pubsubTopic: str
contentTopics: List[str]
fromTime: int
toTime: int
def fetchMissingMessages(peerID, queryParams):
missingMessageHashes = []
# get missing message identifiers first in order to reduce the data transfer
response = waku.store.queryMessageHashes(peerID, queryParams)
for !response.isComplete():
# process each message in the response
response.messages().forEach(messageHash -> {
message = queryDbMessageByHash(messageHash)
if message.exists():
continue
}
missingMessageHashes.append(messageHash)
})
# process next page of the response
response.Next()
# fetch missing messages with hashes in batch
response = waku.store.queryMessagesByHash(peerID, missingMessageHashes)
response.messages().forEach(message -> {
processMessage(message)
})
updateFetchRecord(queryParams.pubsubTopic, queryParams.toTime)
```
`QueryParam` includes all the necessary information to fetch missing messages. The application should iterate all the interested pubsub topics, along with its content topics to construct the `QueryParam`.
Function `fetchMissingMessages` is runing periocally, for example 1 minute. It first fetch all the message hashes in the specified time range, check if message exist in local dabatase, if not, fetch the missing messages in batch. The batch size should be bounded to avoid large data transfer or exceed the max supported size by store node.
When finishing fetching missing messages, the application should update the last fetch time in `FetchRecord`. The last fetch time can be used to calculate the time range for the next fetch and avoid fetching the same messages again.
### Unified Query
There are cases that both outgoing and incoming messages are queried in similar situation, for example at same interval. The application can combine the above two worflows into one to have a unified query with better performance overall.
The workflow can be like this:
- create outgoing buffer for all "outgoing" messages
- create incoming buffer for all recently received message hashes
- periodically query store node based on interested topics and time range for message hashes
- check outgoing buffer with returned message hash, if included, mark message as `sent`, resend if needed
- check incoming buffer with returned message hash, if not included, fetch the missing message with its hash
## Security and Performance Considerations
The message query request exposes the metadata of clients to the store nodes, and the store node is capable to associate the messages with interested clents.
The query requests add a fair amount of load to store nodes, and increased linearly with more users onboarded. Store nodes should be able to scale up and scale down itself by monitoring or predicting the workload.
The store node can also be a target for DDoS attack. The store node should have a mechanism to prevent such attack.
Application should provide options to configure different store nodes for its users, such nodes can either be self-hosted or public with better reputation.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
1. [Relay Protocol](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
2. [Light Push Protocl](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md)
3. [Filter Protocl](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/12/filter.md)
4. [MVDS - Minimum Viable Data Synchronization](https://github.com/vacp2p/rfc-index/blob/main/vac/2/mvds.md)
5. [End-to-end reliability for scalable distributed logs](https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293)
6. [Waku Store Query](https://github.com/waku-org/specs/blob/master/standards/core/store.md)
7. [Waku v2 Discv5 Ambient Peer Discovery](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md)
8. [Waku v2 Peer Exchange](https://github.com/waku-org/specs/blob/master/standards/core/peer-exchange.md)

13
standards/core/enr.md
View File

@ -8,7 +8,7 @@ contributors:
## Abstract
This RFC describes the usage of the ENR (Ethereum Node Records) format for [10/WAKU2](https://rfc.vac.dev/spec/10/) purposes.
This RFC describes the usage of the ENR (Ethereum Node Records) format for [10/WAKU2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md) purposes.
The ENR format is defined in [EIP-778](https://eips.ethereum.org/EIPS/eip-778) [[3]](#references).
This RFC is an extension of EIP-778, ENR used in Waku v2 MUST adhere to both EIP-778 and 31/WAKU2-ENR.
@ -41,7 +41,7 @@ Would carry some ambiguity: Is the certificate securing the websocket port valid
the ipv4 address?
the ipv6 address?
The [10/WAKU2](https://rfc.vac.dev/spec/10/) protocol family is built on the [libp2p](https://github.com/libp2p/specs) protocol stack.
The [10/WAKU2](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md) protocol family is built on the [libp2p](https://github.com/libp2p/specs) protocol stack.
Hence, it uses [multiaddr](https://github.com/multiformats/multiaddr) to format network addresses.
Directly storing one or several multiaddresses in the ENR would fix the issues listed above:
@ -162,7 +162,8 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References
- [1](https://github.com/status-im/nim-waku/pull/690)
- [2](https://github.com/vacp2p/rfc/issues/462#issuecomment-943869940)
- [3](https://eips.ethereum.org/EIPS/eip-778)
- [4](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)
- [1](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/10/waku2.md)
- [2](https://github.com/status-im/nim-waku/pull/690)
- [3](https://github.com/vacp2p/rfc/issues/462#issuecomment-943869940)
- [4](https://eips.ethereum.org/EIPS/eip-778)
- [5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)

14
standards/core/incentivization.md
View File

@ -84,7 +84,7 @@ Note: this section may later be moved to Store RFC.
Store is one of Waku's request-response protocols.
A Store client queries the server for historic messages.
A Store server responds with a list of messages that pass the user's filter.
See [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) for the definitions of `HistoryQuery` and `HistoryResponse`.
See [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md) for the definitions of `HistoryQuery` and `HistoryResponse`.
The PoC Store incentivization makes the following simplifying assumptions:
@ -216,17 +216,17 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
### normative
- A high-level [incentivization outline](https://github.com/waku-org/research/blob/master/incentivization.md)
- [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) (for Store-specific sections)
- [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md) (for Store-specific sections)
### informative
RFCs of request-response protocols:
- [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12)
- [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/)
- [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/)
- [12/WAKU2-FILTER](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/12/filter.md)
- [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/13/store.md)
- [19/WAKU2-LIGHTPUSH](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/19/lightpush.md)
RFCs of Relay and RLN-Relay:
- [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11)
- [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17)
- [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [17/WAKU2-RLN-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/17/rln-relay.md)

95
standards/core/lightpush.md Normal file
View File

@ -0,0 +1,95 @@
---
title: WAKU-LIGHTPUSH
name: Waku Light Push
editor: Zoltan Nagy <zoltan@status.im>
contributors:
- Hanno Cornelius <hanno@status.im>
- Daniel Kaiser <danielkaiser@status.im>
- Oskar Thorén <oskarth@titanproxy.com>
---
previous version: `/vac/waku/lightpush/2.0.0-beta1` [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/waku/standards/core/19/lightpush)
---
**Protocol identifier**: `/vac/waku/lightpush/3.0.0`
## Motivation and Goals
Light nodes with short connection windows and limited bandwidth wish to push messages to other nodes in the Waku network to request message services.<br>
A common use case is to request that the service node publish the message to an `11/WAKU2-RELAY` pubsub-topic.
Additionally, there is sometimes a need for confirmation that a message has been received "by the network"
(here, at least one node).
`WAKU-LIGHTPUSH` is a request/response protocol for this.
## Payloads
```protobuf
syntax = "proto3";
message LightPushRequest {
string request_id = 1;
// 10 Reserved for future `request_type`. Currently, RELAY is the only available service.
optional string pubsub_topic = 20;
WakuMessage message = 21;
}
message LightPushResponse {
string request_id = 1;
uint32 status_code = 10; // non zero in case of failure, see appendix
optional string status_desc = 11;
optional uint32 relay_peer_count = 12; // number of peers, message is successfully relayed to
}
```
### Message Relaying
Nodes that respond to `LightPushRequest` SHOULD
- either relay the encapsulated message via [11/WAKU2-RELAY](https://rfc.vac.dev/waku/standards/core/11/relay) protocol on the specified `pubsub_topic`<br>
- or perform another requested service.
`Services beyond [11/WAKU2-RELAY](https://rfc.vac.dev/waku/standards/core/11/relay) are as yet underdefined.`
Depending on the network configuration, the lightpush client may not need to provide `pubsub_topic` ([WAKU2-RELAY-SHARDING](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md)).<br>
If the node is unable to perform the request for some reason, it SHOULD return an error code in `LightPushResponse`.<br>
Once the relay is successful, the `relay_peer_count` will indicate the number of peers that the node has managed to relay the message to. It's important to note that this number may vary depending on the node subscriptions and support for the requested pubsub_topic. The client can use this information to either consider the relay as successful or take further action, such as switching to a lightpush service peer with better connectivity.<br>
>The field `relay_peer_count` may not be present or has the value zero in case of error or in other future use cases, where no relay is involved.
### Examples of possible error codes
| Result | Code | Note |
|--------|------|------|
| SUCCESS | 200 | Successfull push, response's relay_peer_count holds the number of peers the message is pushed. |
| BAD_REQUEST | 400 | Wrong request payload. |
| PAYLOAD_TOO_LARGE | 413 | Message exceeds certain size limit, it can depend on network configuration, see status_desc for details. |
| UNSUPPORTED_PUBSUB_TOPIC | 421 | Requested push on pubsub_topic is not possible as the service node does not support it. |
| TOO_MANY_REQUESTS | 429 | DOS protection prevented this request as the current request exceeds the configured request rate. |
| INTERNAL_SERVER_ERROR | 500 | status_desc holds explanation of the error. |
| NO_PEERS_TO_RELAY | 503 | Lightpush service is not available as the node has no relay peers. |
> The list of error codes is not complete and can be extended in the future.
## Security Considerations
Since this can introduce an amplification factor, it is RECOMMENDED for the node relaying to the rest of the network to take extra precautions.
Therefore Waku applies or will apply:
- DOS protection through request rate limitation on the service itself.
- message rate limiting via [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/waku/standards/core/17/rln-relay), applied via network membership subscription.
> These features are under development.
## Future work
- Add support attaching RLN proof for the message requested to be relayed.
- Add support for other request types.
- Incentivization of the service
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
* [11/WAKU2-RELAY](https://rfc.vac.dev/waku/standards/core/11/relay)
* [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/waku/standards/core/17/rln-relay)

37
standards/core/metadata.md
View File

@ -1,37 +0,0 @@
---
title: WAKU2-METADATA
name: Waku Metadata Protocol
editor: Alvaro Revuelta <alrevuelta@status.im>
contributors:
---
## Metadata Protocol
Waku specifies a req/resp protocol that provides information about the node's medatadata. Such metadata is meant to be used
by the node to decide if a peer is worth connecting or not. The node that makes the request, includes its metadata
so that the receiver is aware of it, without requiring an extra interaction. The parameters are the following:
* `clusterId`: Unique identifier of the cluster that the node is running in.
* `shards`: Shard indexes that the node is subscribed to.
### Protocol id
`/vac/waku/metadata/1.0.0`
### Request
```proto
message WakuMetadataRequest {
optional uint32 cluster_id = 1;
repeated uint32 shards = 2;
}
```
### Response
```proto
message WakuMetadataResponse {
optional uint32 cluster_id = 1;
repeated uint32 shards = 2;
}
```

311
standards/core/network.md
View File

@ -1,311 +0,0 @@
---
title: WAKU2-NETWORK
name: Waku v2 Network
category: Best Current Practice
editor: Hanno Cornelius <hanno@status.im>
contributors:
---
## Abstract
This RFC specifies an opinionated deployment of [10/WAKU2](https://rfc.vac.dev/spec/10/) protocols
to form a coherent and shared decentralized messaging network
that is open-access,
useful for generalized messaging,
privacy-preserving,
scalable and
accessible even to resource-restricted devices.
We'll refer to this opinionated deployment simply as
_the public Waku Network_, _the Waku Network_ or, if the context is clear, _the network_
in the rest of this document.
## Theory / Semantics
### Routing protocol
The Waku Network is built on the [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) routing protocol,
which in turn is an extension of [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/) with spam protection measures.
### Network shards
Traffic in the Waku Network is sharded into eight [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) pubsub topics.
Each pubsub topic is named according to the static shard naming format
defined in [WAKU2-RELAY-SHARDING](./relay-sharding.md)
with:
- `<cluster_id>` set to `1`
- `<shard_number>` occupying the range `0` to `7`.
In other words, the Waku Network is a [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) network
routed on the combination of the eight pubsub topics:
```
/waku/2/rs/1/0
/waku/2/rs/1/1
...
/waku/2/rs/1/7
```
A node MUST use [WAKU-METADATA](./metadata.md) protocol to identify the `<cluster_id>` that every
inbound/outbound peer that attempts to connect supports. In any of the following cases, the node MUST trigger a disconnection:
- [WAKU-METADATA](./metadata.md) dial fails.
- [WAKU-METADATA](./metadata.md) reports an empty `<cluster_id>`.
- [WAKU-METADATA](./metadata.md) reports a `<cluster_id>` different than `1`.
## Roles
There are two distinct roles evident in the network, those of:
1. nodes, and
2. applications.
### Nodes
Nodes are the individual software units
using [10/WAKU2](https://rfc.vac.dev/spec/10/) protocols to form a p2p messaging network.
Nodes, in turn, can participate in a shard as full relayers, i.e. _relay nodes_,
or by running a combination of protocols suitable for resource-restricted environments, i.e. _non-relay nodes_.
Nodes can also provide various services to the network,
such as storing historical messages or protecting the network against spam.
See the section on [default services](#default-services) for more.
#### Relay nodes
Relay nodes MUST follow [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/)
to route messages to other nodes in the network
for any of the pubsub topics [defined as the Waku Network shards](#network-shards).
Relay nodes MAY choose to subscribe to any of these shards,
but MUST be subscribed to at least one defined shard.
Each relay node SHOULD be subscribed to as many shards as it has resources to support.
If a relay node supports an encapsulating application,
it SHOULD be subscribed to all the shards servicing that application.
If resource restrictions prevent a relay node from servicing all shards used by the encapsulating application,
it MAY choose to support some shards as a non-relay node.
#### Bootstrapping and discovery
Nodes MAY use any method to bootstrap connection to the network,
but it is RECOMMENDED that each node retrieves a list of bootstrap peers to connect to using [EIP-1459 DNS-based discovery](https://eips.ethereum.org/EIPS/eip-1459).
Relay nodes SHOULD use [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) to continually discover other peers in the network.
Each relay node MUST encode its supported shards into its discoverable ENR
as described in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#discovery).
The ENR MUST be updated if the set of supported shards change.
A node MAY choose to ignore discovered peers that do not support any of the shards in its own subscribed set.
#### Transports
Relay nodes MUST follow [10/WAKU2](https://rfc.vac.dev/spec/10/) specifications with regards to supporting different transports.
If TCP transport is available, each relay node MUST support it as transport for both dialing and listening.
In addition, a relay node SHOULD support secure websockets for bidirectional communication streams,
for example to allow connections from and to web browser-based clients.
A relay node MAY support unsecure websockets if required by the application or running environment.
#### Default services
For each supported shard,
each relay node SHOULD enable and support the following protocols as a service node:
1. [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12/) to allow resource-restricted peers to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) to allow other peers to request historical messages from this node.
3. [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) to allow resource-restricted peers to request publishing a message to the network on their behalf.
4. [34/WAKU2-PEER-EXCHANGE](./peer-exchange.md) to allow resource-restricted peers to discover more peers in a resource efficient way.
#### Store service nodes
Each relay node SHOULD support [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) as a store service node,
for each supported shard.
The store SHOULD be configured to retain at least `12` hours of messages per supported shard.
Store service nodes SHOULD only store messages with a valid [`rate_limit_proof`](#message-attributes) attribute.
#### Non-relay nodes
Nodes MAY opt out of relay functionality on any network shard
and instead request services from relay nodes as clients
using any of the defined service protocols:
1. [12/WAKU2-FILTER](https://rfc.vac.dev/spec/12/) to subscribe to messages matching a specific content filter.
2. [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) to request historical messages matching a specific content filter.
3. [19/WAKU2-LIGHTPUSH](https://rfc.vac.dev/spec/19/) to request publishing a message to the network.
4. [34/WAKU2-PEER-EXCHANGE](./peer-exchange.md) to discover more peers in a resource efficient way.
#### Store client nodes
Nodes MAY request historical messages from [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) service nodes as store clients.
A store client SHOULD discard any messages retrieved from a store service node that do not contain a valid [`rate_limit_proof`](#message-attributes) attribute.
The client MAY consider service nodes returning messages without a valid [`rate_limit_proof`](#message-attributes) attribute as untrustworthy.
The mechanism by which this may happen is currently underdefined.
### Applications
Applications are the higher-layer projects or platforms that make use of the generalized messaging capability of the network.
In other words, an application defines a payload used in the various [10/WAKU2](https://rfc.vac.dev/spec/10/) protocols.
Any participant in an application SHOULD make use of an underlying node in order to communicate on the network.
Applications SHOULD make use of an [autosharding](#autosharding) API
to allow the underlying node to automatically select the target shard on the Waku Network. See the section on [autosharding](#autosharding) for more.
## RLN rate-limiting
The [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/) network uses [32/RLN-V1](https://rfc.vac.dev/spec/32/) proofs
to ensure that a pre-agreed rate limit is not exceeded by any publisher.
While the network is under capacity,
individual relayers MAY choose to freely route messages without RLN proofs
up to a discretionary bandwidth limit after which messages without proofs MUST be discarded.
This bandwidth limit SHOULD be enforced using [bandwidth validation mechanism](#free-bandwidth-exceeded) separate from RLN rate-limiting.
This implies that quality of service and reliability is significantly lower for messages without proofs
and at times of high network utilization these messages may not be relayed at all.
### RLN Parameters
For the Waku Network,
the `epoch` is set to `1` second
and the maximum number of messages published per `epoch` is limited to `1` per publisher.
The `max_epoch_gap` is set to `20` seconds,
meaning that validators MUST _reject_ messages with an `epoch` more than 20 seconds into the past or future compared to the validator's own clock.
All nodes, validators and publishers,
SHOULD use Network Time Protocol (NTP) to synchronize their own clocks,
thereby ensuring valid timestamps for proof generation and validation.
### Memberships
Each publisher to the Waku Network SHOULD register an RLN membership
with one of the RLN storage contracts
moderated in the Sepolia registry contract with address [0xF1935b338321013f11068abCafC548A7B0db732C](https://sepolia.etherscan.io/address/0xF1935b338321013f11068abCafC548A7B0db732C#code).
Initial memberships are registered in the Sepolia RLN storage contract with address [0x58322513A35a8f747AF5A385bA14C2AbE602AA59](https://sepolia.etherscan.io/address/0x58322513A35a8f747AF5A385bA14C2AbE602AA59#code).
RLN membership setup and registration MUST follow [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/#setup-and-registration),
with the `staked_fund` set to `0`.
In other words, the Waku Network does not use RLN staking.
### RLN Proofs
Each RLN member MUST generate and attach an RLN proof to every published message
as described in [17/WAKU-RLN-RELAY](https://rfc.vac.dev/spec/17/#publishing).
Slashing is not implemented for the Waku Network.
Instead, validators will penalise peers forwarding messages exceeding the rate limit
as specified for [the rate-limiting validation mechanism](#rate-limit-exceeded).
This incentivizes all nodes to validate RLN proofs
and reject messages violating rate limits
in order to continue participating in the network.
## Network traffic
All payload on the Waku Network MUST be encapsulated in a [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/)
with rate limit proof extensions defined for [17/WAKU2-RLN-RELAY](https://rfc.vac.dev/spec/17/#payloads).
Each message on the Waku Network SHOULD be validated by each relayer,
according to the rules discussed under [message validation](#message-validation).
### Message Attributes
- The mandatory `payload` attribute MUST contain the message data payload as crafted by the application.
- The mandatory `content_topic` attribute MUST specify a string identifier that can be used for content-based filtering. This is also crafted by the application. See [Autosharding](#autosharding) for more on the content topic format.
- The optional `meta` attribute MAY be omitted. If present this will form part of the message uniqueness vector described in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/).
- The optional `version` attribute SHOULD be set to `0`. It MUST be interpreted as `0` if not present.
- The mandatory `timestamp` attribute MUST contain the Unix epoch time at which the message was generated by the application. The value MUST be in nanoseconds. It MAY contain a fudge factor of up to 1 seconds in either direction to improve resistance to timing attacks.
- The optional `ephemeral` attribute MUST be set to `true` if the message should not be persisted by the Waku Network.
- The optional `rate_limit_proof` attribute SHOULD be populated with the RLN proof as set out in [RLN Proofs](#rln-proofs). Messages with this field unpopulated MAY be discarded from the network by relayers. This field MUST be populated if the message should be persisted by the Waku Network.
### Message Size
Any Waku Message published to the network MUST NOT exceed an absolute maximum size of `150` kilobytes.
This limit applies to the entire message after protobuf serialization, including attributes.
It is RECOMMENDED not to exceed an average size of `4` kilobytes for Waku Messages published to the network.
### Message Validation
Relay nodes MUST apply [gossipsub v1.1 validation](https://github.com/libp2p/specs/blob/c96c9ec5909d64fe020d7630f3fd982bc18fd06a/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) to each relayed message and
SHOULD apply all of the rules set out in the section below to determine the validity of a message.
Validation has one of three outcomes,
repeated here from the [gossipsub specification](https://github.com/libp2p/specs/blob/c96c9ec5909d64fe020d7630f3fd982bc18fd06a/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) for ease of reference:
1. Accept - the message is considered valid and it MUST be delivered and forwarded to the network.
2. Reject - the message is considered invalid, MUST be rejected and SHOULD trigger a gossipsub scoring penalty against the transmitting peer.
3. Ignore - the message SHOULD NOT be delivered and forwarded to the network, but this MUST NOT trigger a gossipsub scoring penalty against the transmitting peer.
The following validation rules are defined:
#### Decoding failure
If a message fails to decode as a valid [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/).
the relay node MUST _reject_ the message.
This SHOULD trigger a penalty against the transmitting peer.
#### Invalid timestamp
If a message has a timestamp deviating by more than `20` seconds
either into the past or the future
when compared to the relay node's internal clock,
the relay node MUST _reject_ the message.
This allows for some deviation between internal clocks,
network routing latency and
an optional [fudge factor when timestamping new messages](#message-attributes).
#### Free bandwidth exceeded
If a message contains no RLN proof
and the current bandwidth utilization on the shard the message was published to
equals or exceeds `1` Mbps,
the relay node SHOULD _ignore_ the message.
#### Invalid RLN epoch
If a message contains an RLN proof
and the `epoch` attached to the proof deviates by more than `max_epoch_gap` seconds
from the relay node's own `epoch`,
the relay node MUST _reject_ the message.
`max_epoch_gap` is [set to `20` seconds](#rln-parameters) for the Waku Network.
#### Invalid RLN proof
If a message contains an RLN proof
and the zero-knowledge proof is invalid
according to the verification process described in [32/RLN-V1](https://rfc.vac.dev/spec/32/#verification),
the relay node MUST _ignore_ the message.
#### Rate limit exceeded
If a message contains an RLN proof
and the relay node detects double signaling
according to the verification process described in [32/RLN-V1](https://rfc.vac.dev/spec/32/#verification),
the relay node MUST _reject_ the message
for violating the agreed rate limit of `1` message every `1` second.
This SHOULD trigger a penalty against the transmitting peer.
## Autosharding
Nodes in the Waku Network SHOULD allow encapsulating applications to use autosharding,
as defined in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#automatic-sharding)
by automatically determining the appropriate pubsub topic
from the list [of defined Waku Network shards](#network-shards).
This allows the application to omit the target pubsub topic
when invoking any Waku protocol function.
Applications using autosharding MUST use content topics in the format
defined in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#content-topics-format-for-autosharding)
and SHOULD use the short length format:
```
/{application-name}/{version-of-the-application}/{content-topic-name}/{encoding}`
```
When an encapsulating application makes use of autosharding
the underlying node MUST determine the target pubsub topic(s)
from the content topics provided by the application
using the hashing mechanism defined in [51/WAKU2-RELAY-SHARDING](https://rfc.vac.dev/spec/51/#automatic-sharding).
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
## References
- [WAKU2-RELAY-SHARDING](./relay-sharding.md)
- [Peer-exchange](./peer-exchange.md)
## normative
(TBD)
A list of references that MUST be read to fully understand and/or implement this protocol.
See [RFC3967 Section 1.1](https://datatracker.ietf.org/doc/html/rfc3967#section-1.1).
## informative
(TBD)
A list of additional references.

21
standards/core/relay-sharding.md
View File

@ -11,7 +11,7 @@ contributors:
## Abstract
This document describes ways of sharding the [Waku relay](https://rfc.vac.dev/spec/11/) topic,
This document describes ways of sharding the [Waku relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) topic,
allowing Waku networks to scale in the number of content topics.
> _Note_: Scaling in the size of a single content topic is out of scope for this document.
@ -25,7 +25,7 @@ However, they do not scale to large traffic loads.
A single [libp2p gossipsub mesh](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#gossipsub-the-gossiping-mesh-router),
which carries messages associated with a single pubsub topic, can be seen as a separate unstructured P2P network
(control messages go beyond these boundaries, but at its core, it is a separate P2P network).
With this, the number of [Waku relay](https://rfc.vac.dev/spec/11/) content topics that can be carried over a pubsub topic is limited.
With this, the number of [Waku relay](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md) content topics that can be carried over a pubsub topic is limited.
This prevents app protocols that aim to span many multicast groups (realized by content topics) from scaling.
This document specifies three pubsub topic sharding methods (with varying degrees of automation),
@ -35,7 +35,7 @@ This document also covers discovery of topic shards.
## Named Sharding
_Named sharding_ offers apps to freely choose pubsub topic names.
It is RECOMMENDED for App protocols to follow the naming structure detailed in [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/).
It is RECOMMENDED for App protocols to follow the naming structure detailed in [23/WAKU2-TOPICS](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/23/topics.md).
With named sharding, managing discovery falls into the responsibility of apps.
From an app protocol point of view, a subscription to a content topic `waku2/xxx` on a shard named /mesh/v1.1.1/xxx would look like:
@ -82,7 +82,7 @@ an example for the 2nd shard in the global shard cluster:
`/waku/2/rs/0/2`.
> _Note_: Because _all_ shards distribute payload defined in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14/) via [protocol buffers](https://developers.google.com/protocol-buffers/),
> _Note_: Because _all_ shards distribute payload defined in [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md) via [protocol buffers](https://developers.google.com/protocol-buffers/),
> the pubsub topic name does not explicitly add `/proto` to indicate protocol buffer encoding.
> We use `rs` to indicate these are _relay shard_ clusters; further shard types might follow in the future.
@ -102,7 +102,7 @@ so app protocols do not have to implement their own discovery method.
Nodes add information about their shard participation in their [WAKU2-ENR](./enr.md).
Having a static shard participation indication as part of the ENR allows nodes
to discover peers that are part of shards via [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) as well as via DNS.
to discover peers that are part of shards via [33/WAKU2-DISCV5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md) as well as via DNS.
> _Note:_ In the current version of this document,
> sharding information is directly added to the ENR.
@ -191,7 +191,7 @@ The shard to use is the modulo of the hash by the number of shards in the networ
### Content Topics Format for Autosharding
Content topics MUST follow the format in [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/#content-topic-format).
Content topics MUST follow the format in [23/WAKU2-TOPICS](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/23/topics.md/#content-topic-format).
In addition, a generation prefix MAY be added to content topics.
When omitted default values are used.
Generation default value is `0`.
@ -242,7 +242,7 @@ but is planned to preserve the index range allocation.
Instead of adding the data to the ENR, it will treat each array index as a capability,
which can be hierarchical, having each shard in the indexed shard cluster as a sub-capability.
When scaling to a very large number of shards, this will avoid blowing up the ENR size, and allows efficient discovery.
We currently use [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/) for discovery,
We currently use [33/WAKU2-DISCV5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md) for discovery,
which is based on Ethereum's [discv5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md).
While this allows to sample nodes from a distributed set of nodes efficiently and offers good resilience,
it does not allow to efficiently discover nodes with specific capabilities within this node set.
@ -272,12 +272,13 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
## References
- [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/)
- [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/11/relay.md)
- [Unstructured P2P network](https://en.wikipedia.org/wiki/Peer-to-peer#Unstructured_networks)
- [33/WAKU2-DISCV5](https://rfc.vac.dev/spec/33/)
- [33/WAKU2-DISCV5](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/33/discv5.md)
- [WAKU2-ENR](./enr.md)
- [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/)
- [23/WAKU2-TOPICS](https://github.com/vacp2p/rfc-index/blob/main/waku/informational/23/topics.md)
- [Ethereum ENR sharding bit vector](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#metadata)
- [Ethereum discv5 specification](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md)
- [Research log: Waku Discovery](https://vac.dev/wakuv2-apd)
- [WAKU2-RELAY-STATIC-SHARD-ALLOC](../../informational/relay-static-shard-alloc.md)
- [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/14/message.md)

503
standards/core/store.md
View File

@ -1,208 +1,330 @@
---
title: 13/WAKU2-STORE
name: Waku v2 Store
status: draft
tags: [waku-core]
editor: Sanaz Taheri <sanaz@status.im>
title: WAKU2-STORE
name: Waku Store Query
editor: Hanno Cornelius <hanno@status.im>
contributors:
- Dean Eigenmann <dean@status.im>
- Oskar Thorén <oskarth@titanproxy.com>
- Aaryamann Challani <aaryamann@status.im>
- Sanaz Taheri <sanaz@status.im>
---
> **Note:** This version of WAKU2-STORE is earmarked to replace RFC [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/13/store.md) once it reaches draft status
---
# Abstract
This specification explains the `13/WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
stored by other nodes.
It also supports pagination for more efficient querying of historical messages.
This specification explains the `WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
stored by other nodes.
It also supports pagination for more efficient querying of historical messages.
**Protocol identifier\***: `/vac/waku/store/2.0.0-beta4`
**Protocol identifier***: `/vac/waku/store-query/3.0.0`
## Terminology
The term PII, Personally Identifiable Information,
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
The term PII, Personally Identifiable Information,
refers to any piece of data that can be used to uniquely identify a user.
For example, the signature verification key, and
the hash of one's static IP address are unique for each user and hence count as PII.
# Design Requirements
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
Nodes willing to provide the storage service using `WAKU2-STORE` protocol,
SHOULD provide a complete and full view of message history.
As such, they are required to be _highly available_ and
specifically have a _high uptime_ to consistently receive and store network messages.
The high uptime requirement makes sure that no message is missed out hence a complete and
As such, they are required to be *highly available* and
specifically have a *high uptime* to consistently receive and store network messages.
The high uptime requirement makes sure that no message is missed out hence a complete and
intact view of the message history is delivered to the querying nodes.
Nevertheless, in case storage provider nodes cannot afford high availability,
Nevertheless, in case storage provider nodes cannot afford high availability,
the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
The concept of `ephemeral` messages introduced in [`14/WAKU2-MESSAGE`](https://rfc.vac.dev/spec/14) affects `13/WAKU2-STORE` as well.
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
The concept of `ephemeral` messages introduced in [`WAKU2-MESSAGE`](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) affects `WAKU2-STORE` as well.
Nodes running `WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md).
Nodes running `WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
# Adversarial Model
Any peer running the `13/WAKU2-STORE` protocol, i.e.
both the querying node and the queried node, are considered as an adversary.
Furthermore,
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
it does so without violating protocol definitions and instructions.
As we evolve the protocol,
Any peer running the `WAKU2-STORE` protocol, i.e.
both the querying node and the queried node, are considered as an adversary.
Furthermore,
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
it does so without violating protocol definitions and instructions.
As we evolve the protocol,
further adversarial models will be considered.
For example, under the passive adversarial model,
no malicious node hides or
lies about the history of messages as it is against the description of the `13/WAKU2-STORE` protocol.
For example, under the passive adversarial model,
no malicious node hides or
lies about the history of messages as it is against the description of the `WAKU2-STORE` protocol.
The following are not considered as part of the adversarial model:
- An adversary with a global view of all the peers and their connections.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
In specific, the communication channels are assumed to be secure.
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
In specific, the communication channels are assumed to be secure.
# Wire Specification
Peers communicate with each other using a request / response API.
The messages sent are Protobuf RPC messages which are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
The following are the specifications of the Protobuf messages.
## Payloads
```protobuf
syntax = "proto3";
message Index {
bytes digest = 1;
sint64 receiverTime = 2;
sint64 senderTime = 3;
string pubsubTopic = 4;
// Protocol identifier: /vac/waku/store-query/3.0.0
package waku.store.v3;
import "waku/message/v1/message.proto";
message WakuMessageKeyValue {
optional bytes message_hash = 1; // Globally unique key for a Waku Message
// Full message content and associated pubsub_topic as value
optional waku.message.v1.WakuMessage message = 2;
optional string pubsub_topic = 3;
}
message PagingInfo {
uint64 pageSize = 1;
Index cursor = 2;
enum Direction {
BACKWARD = 0;
FORWARD = 1;
}
Direction direction = 3;
}
message ContentFilter {
string contentTopic = 1;
}
message HistoryQuery {
// the first field is reserved for future use
string pubsubtopic = 2;
repeated ContentFilter contentFilters = 3;
PagingInfo pagingInfo = 4;
}
message HistoryResponse {
// the first field is reserved for future use
repeated WakuMessage messages = 2;
PagingInfo pagingInfo = 3;
enum Error {
NONE = 0;
INVALID_CURSOR = 1;
}
Error error = 4;
}
message HistoryRPC {
message StoreQueryRequest {
string request_id = 1;
HistoryQuery query = 2;
HistoryResponse response = 3;
bool include_data = 2; // Response should include full message content
// Filter criteria for content-filtered queries
optional string pubsub_topic = 10;
repeated string content_topics = 11;
optional sint64 time_start = 12;
optional sint64 time_end = 13;
// List of key criteria for lookup queries
repeated bytes message_hashes = 20; // Message hashes (keys) to lookup
// Pagination info. 50 Reserved
optional bytes pagination_cursor = 51; // Message hash (key) from where to start query (exclusive)
bool pagination_forward = 52;
optional uint64 pagination_limit = 53;
}
message StoreQueryResponse {
string request_id = 1;
optional uint32 status_code = 10;
optional string status_desc = 11;
repeated WakuMessageKeyValue messages = 20;
optional bytes pagination_cursor = 51;
}
```
## General store query concepts
### Index
### Waku message key-value pairs
To perform pagination,
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is associated with a unique `Index` that encapsulates the following parts.
The store query protocol operates as a query protocol for a key-value store of historical Waku messages,
with each entry having a [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) and associated pubsub topic as value,
and [deterministic message hash](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md#deterministic-message-hashing) as key.
The store can be queried to return either a set of keys or a set of key-value pairs.
Within the store query protocol, Waku message keys and values MUST be represented in a `WakuMessageKeyValue` message.
This message MUST contain the deterministic `message_hash` as key.
It MAY contain the full `WakuMessage` and associated pubsub topic as value in the `message` and `pubsub_topic` fields,
depending on the use case as set out below.
If the message contains a value entry in addition to the key,
both the `message` and `pubsub_topic` fields MUST be populated.
The message MUST NOT have either `message` or `pubsub_topic` populated with the other unset.
Both fields MUST either be set or unset.
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
The hash is computed over the concatenation of `contentTopic` and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14)).
- `receiverTime`: the UNIX time in nanoseconds at which the `WakuMessage` is received by the receiving node.
- `senderTime`: the UNIX time in nanoseconds at which the `WakuMessage` is generated by its sender.
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
### Waku message store eligibility
### PagingInfo
In order for a Waku message to be eligible for storage:
- it MUST be a _valid_ [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md).
- the `timestamp` field MUST be populated with the Unix epoch time at which the message was generated in nanoseconds.
If at the time of storage the `timestamp` deviates by more than 20 seconds
either into the past or the future when compared to the store node’s internal clock,
the store node MAY reject the message.
- the `ephemeral` field MUST be set to `false`.
`PagingInfo` holds the information required for pagination. It consists of the following components.
### Waku message sorting
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery`
(or retrieved `WakuMessage`s in a `HistoryResponse`).
- `cursor`: holds the `Index` of a `WakuMessage`.
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
The key-value entries in the store MUST be time-sorted by the `WakuMessage` `timestamp` attribute.
Where two or more key-value entries have identical `timestamps`,
the entries MUST be further sorted by the natural order of their message hash keys.
Within the context of traversing over key-value entries in the store,
_"forward"_ indicates traversing the entries in ascending order,
whereas _"backward"_ indicates traversing the entries in descending order.
### ContentFilter
### Pagination
`ContentFilter` carries the information required for filtering historical messages.
If a large number of entries in the store service node match the query criteria provided in a `StoreQueryRequest`,
the client MAY make use of pagination
in a chain of store query request and response transactions
to retrieve the full response in smaller batches termed _"pages"_.
Pagination can be performed either in [a _forward_ or _backward_ direction](#waku-message-sorting).
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
by setting the page size limit in the `pagination_limit` field.
Note that a store service node MAY enforce its own limit
if the `pagination_limit` is unset
or larger than the service node's internal page size limit.
### HistoryQuery
A `StoreQueryResponse` with a populated `pagination_cursor` indicates that more stored entries match the query than included in the response.
RPC call to query historical messages.
A `StoreQueryResponse` without a populated `pagination_cursor` indicates that
there are no more matching entries in the store.
- The `pubsubTopic` field MUST indicate the pubsub topic of the historical messages to be retrieved.
This field denotes the pubsub topic on which `WakuMessage`s are published.
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/).
Leaving this field empty means no filter on the pubsub topic of message history is requested.
This field SHOULD be left empty in order to retrieve the historical `WakuMessage` regardless of the pubsub topics on which they are published.
- The `contentFilters` field MUST indicate the list of content filters based on which the historical messages are to be retrieved.
Leaving this field empty means no filter on the content topic of message history is required.
This field SHOULD be left empty in order to retrieve historical `WakuMessage` regardless of their content topics.
- `PagingInfo` holds the information required for pagination.
Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`.
It is RECOMMENDED that the queried node defines a maximum page size internally.
If the querying node leaves the `pageSize` unspecified,
or if the `pageSize` exceeds the maximum page size,
the queried node SHOULD auto-paginate the `HistoryResponse` to no more than the configured maximum page size.
This allows mitigation of long response time for `HistoryQuery`.
In the forward pagination request,
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
the `pageSize` amount of `WakuMessage` whose `Index` values are larger than the given `cursor`
(and vise versa for the backward pagination).
Note that the `cursor` of a `HistoryQuery` MAY be empty (e.g., for the initial query), as such, and
depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned, respectively.
The client MAY request the next page of entries from the store service node
by populating a subsequent `StoreQueryRequest` with the `pagination_cursor` received in the `StoreQueryResponse`.
All other fields and query criteria MUST be the same as in the preceding `StoreQueryRequest`.
### Sorting Messages
A `StoreQueryRequest` without a populated `pagination_cursor` indicates that
the client wants to retrieve the "first page" of the stored entries matching the query.
The queried node MUST sort the `WakuMessage` based on their `Index`,
where the `senderTime` constitutes the most significant part and the `digest` comes next, and
then perform pagination on the sorted result.
As such, the retrieved page contains an ordered list of `WakuMessage` from the oldest messages to the most recent one.
Alternatively, the `receiverTime` (instead of `senderTime` ) MAY be used to sort messages during the paging process.
However, it is RECOMMENDED the use of the `senderTime` for sorting as it is invariant and
consistent across all the nodes.
This has the benefit of `cursor` reusability i.e.,
a `cursor` obtained from one node can be consistently used to query from another node.
However, this `cursor` reusability does not hold when the `receiverTime` is utilized as the receiver time is affected by the network delay and
nodes' clock asynchrony.
## Store Query Request
### HistoryResponse
A client node MUST send all historical message queries within a `StoreQueryRequest` message.
This request MUST contain a `request_id`.
The `request_id` MUST be a uniquely generated string.
RPC call to respond to a HistoryQuery call.
If the store query client requires the store service node to include Waku message values in the query response,
it MUST set `include_data` to `true`.
If the store query client requires the store service node to return only message hash keys in the query response,
it SHOULD set `include_data` to `false`.
By default, therefore, the store service node assumes `include_data` to be `false`.
- The `messages` field MUST contain the messages found,
these are [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14) types.
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries.
The `pageSize` indicates the number of returned Waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`).
The `direction` is the same direction as in the corresponding `HistoryQuery`.
In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging).
Regardless of the paging direction, the retrieved `messages` are always sorted in ascending order based on their timestamp as explained in the [sorting messages](#sorting-messages) section, that is, from the oldest to the most recent.
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result may be different.
- The `error` field contains information about any error that has occurred while processing the corresponding `HistoryQuery`.
`NONE` stands for no error.
This is also the default value.
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery` does not match with the `Index` of any of the `WakuMessage` persisted by the queried node.
A store query client MAY include query filter criteria in the `StoreQueryRequest`.
There are two types of filter use cases:
1. Content filtered queries and
2. Message hash lookup queries
### Content filtered queries
A store query client MAY request the store service node to filter historical entries by a content filter.
Such a client MAY create a filter on content topic, on time range or on both.
To filter on content topic, the client MUST populate _both_ the `pubsub_topic` _and_ `content_topics` field.
The client MUST NOT populate either `pubsub_topic` or `content_topics` and leave the other unset.
Both fields MUST either be set or unset.
A mixed content topic filter with just one of either `pubsub_topic` or `content_topics` set, SHOULD be regarded as an invalid request.
To filter on time range, the client MUST set `time_start`, `time_end` or both.
Each `time_` field should contain a Unix epoch timestamp in nanoseconds.
An unset `time_start` SHOULD be interpreted as "from the oldest stored entry".
An unset `time_end` SHOULD be interpreted as "up to the youngest stored entry".
If any of the content filter fields are set,
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`,
the client MUST NOT set the `message_hashes` field.
### Message hash lookup queries
A store query client MAY request the store service node to filter historical entries by one or more matching message hash keys.
This type of query acts as a "lookup" against a message hash key or set of keys already known to the client.
In order to perform a lookup query, the store query client MUST populate the `message_hashes` field with the list of message hash keys it wants to lookup in the store service node.
If the `message_hashes` field is set,
the client MUST NOT set any of the content filter fields,
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`.
### Presence queries
A presence query is a special type of lookup query that allows a client to check for the presence of one or more messages in the store service node,
without retrieving the full contents (values) of the messages.
This can, for example, be used as part of a reliability mechanism,
whereby store query clients verify that previously published messages have been successfully stored.
In order to perform a presence query,
the store query client MUST populate the `message_hashes` field in the `StoreQueryRequest` with the list of message hashes
for which it wants to verify presence in the store service node.
The `include_data` property MUST be set to `false`.
The client SHOULD interpret every `message_hash` returned in the `messages` field of the `StoreQueryResponse` as present in the store.
The client SHOULD assume that all other message hashes included in the original `StoreQueryRequest` but not in the `StoreQueryResponse` is not present in the store.
### Pagination info
The store query client MAY include a message hash as `pagination_cursor`,
to indicate at which key-value entry a store service node SHOULD start the query.
The `pagination_cursor` is treated as exclusive
and the corresponding entry will not be included in subsequent store query responses.
For forward queries, only messages following (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor` will be returned.
For backward queries, only messages preceding (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor` will be returned.
If the store query client requires the store service node to perform a forward query,
it MUST set `pagination_forward` to `true`.
If the store query client requires the store service node to perform a backward query,
it SHOULD set `pagination_forward` to `false`.
By default, therefore, the store service node assumes pagination to be backward.
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
by setting the page size limit in the `pagination_limit` field.
Note that a store service node MAY enforce its own limit
if the `pagination_limit` is unset
or larger than the service node's internal page size limit.
See [pagination](#pagination) for more on how the pagination info is used in store transactions.
## Store Query Response
In response to any `StoreQueryRequest`,
a store service node SHOULD respond with a `StoreQueryResponse` with a `requestId` matching that of the request.
This response MUST contain a `status_code` indicating if the request was successful or not.
Successful status codes are in the `2xx` range.
Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
In addition, the store service node MAY choose to provide a more detailed status description in the `status_desc` field.
### Filter matching
For [content filtered queries](#content-filtered-queries), an entry in the store service node matches the filter criteria in a `StoreQueryRequest` if each of the following conditions are met:
- its `content_topic` is in the request `content_topics` set
and it was published on a matching `pubsub_topic` OR the request `content_topics` and `pubsub_topic` fields are unset
- its `timestamp` is _larger or equal_ than the request `start_time` OR the request `start_time` is unset
- its `timestamp` is _smaller_ than the request `end_time` OR the request `end_time` is unset
Note that for content filtered queries, `start_time` is treated as _inclusive_ and `end_time` is treated as _exclusive_.
For [message hash lookup queries](#message-hash-lookup-queries), an entry in the store service node matches the filter criteria if its `message_hash` is in the request `message_hashes` set.
The store service node SHOULD respond with an error code and discard the request
if the store query request contains both content filter criteria and message hashes.
### Populating response messages
The store service node SHOULD populate the `messages` field in the response
only with entries matching the filter criteria provided in the corresponding request.
Regardless of whether the response is to a _forward_ or _backward_ query,
the `messages`field in the response MUST be ordered in a forward direction
according to the [message sorting rules](#waku-message-sorting).
If the corresponding `StoreQueryRequest` has `include_data` set to true,
the service node SHOULD populate both the `message_hash` and `message` for each entry in the response.
In all other cases, the store service node SHOULD populate only the `message_hash` field for each entry in the response.
### Paginating the response
The response SHOULD NOT contain more `messages` than the `pagination_limit` provided in the corresponding `StoreQueryRequest`.
It is RECOMMENDED that the store node defines its own maximum page size internally.
If the `pagination_limit` in the request is unset,
or exceeds this internal maximum page size,
the store service node SHOULD ignore the `pagination_limit` field and apply its own internal maximum page size.
In response to a _forward_ `StoreQueryRequest`:
- if the `pagination_cursor` is set,
the store service node SHOULD populate the `messages` field
with matching entries following the `pagination_cursor` (exclusive).
- if the `pagination_cursor` is unset,
the store service node SHOULD populate the `messages` field
with matching entries from the first entry in the store.
- if there are still more matching entries in the store
after the maximum page size is reached while populating the response,
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
with the message hash key of the _last_ entry _included_ in the response.
In response to a _backward_ `StoreQueryRequest`:
- if the `pagination_cursor` is set,
the store service node SHOULD populate the `messages` field
with matching entries preceding the `pagination_cursor` (exclusive).
- if the `pagination_cursor` is unset,
the store service node SHOULD populate the `messages` field
with matching entries from the last entry in the store.
- if there are still more matching entries in the store
after the maximum page size is reached while populating the response,
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
with the message hash key of the _first_ entry _included_ in the response.
# Security Consideration
@ -211,53 +333,52 @@ The main security consideration to take into account while using this protocol i
# Future Work
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
without disclosing the exact topics of [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14) they are interested in.
As such, no adversary in the `13/WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
content topics of [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity for historical queries,
as the querying node needs to directly connect to another node in the `13/WAKU2-STORE` protocol and
explicitly disclose the content filters of its interest to retrieve the corresponding messages.
However, one can consider preserving anonymity through one of the following ways:
without disclosing the exact topics of [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) they are interested in.
As such, no adversary in the `WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
content topics of [14/WAKU2-MESSAGE](/spec/14).
The current version of the `WAKU2-STORE` protocol does not provide anonymity for historical queries,
as the querying node needs to directly connect to another node in the `WAKU2-STORE` protocol and
explicitly disclose the content filters of its interest to retrieve the corresponding messages.
However, one can consider preserving anonymity through one of the following ways:
- By hiding the source of the request i.e., anonymous communication.
That is the querying node shall hide all its PII in its history request e.g., its IP address.
This can happen by the utilization of a proxy server or by using Tor.
This can happen by the utilization of a proxy server or by using Tor.
Note that the current structure of historical requests does not embody any piece of PII, otherwise,
such data fields must be treated carefully to achieve query anonymity.
<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the store protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a querying node and its queried topics despite hiding the IP address-->
such data fields must be treated carefully to achieve query anonymity.
<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the store protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a querying node and its queried topics despite hiding the IP address-->
- By deploying secure 2-party computations in which the querying node obtains the historical messages of a certain topic,
the queried node learns nothing about the query.
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
<!-- TODO: add a reference for PSIs? --> <!-- TODO: more techniques to be included -->
<!-- TODO: Censorship resistant: this is about a node that hides the historical messages from other nodes. This attack is not included in the specs since it does not fit the passive adversarial model (the attacker needs to deviate from the store protocol).-->
the queried node learns nothing about the query.
Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI).
<!-- TODO: add a reference for PSIs? --> <!-- TODO: more techniques to be included -->
<!-- TODO: Censorship resistant: this is about a node that hides the historical messages from other nodes. This attack is not included in the specs since it does not fit the passive adversarial model (the attacker needs to deviate from the store protocol).-->
- **Robust and verifiable timestamps**: Messages timestamp is a way to show that the message existed prior to some point in time.
However, the lack of timestamp verifiability can create room for a range of attacks,
including injecting messages with invalid timestamps pointing to the far future.
To better understand the attack,
consider a store node whose current clock shows `2021-01-01 00:00:30` (and assume all the other nodes have a synchronized clocks +-20seconds).
The store node already has a list of messages,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
that are sorted based on their timestamp.
An attacker sends a message with an arbitrary large timestamp e.g.,
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
The store node places `m'` at the end of the list,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m',2021-01-01 10:00:30)`.
Now another message arrives with a valid timestamp e.g.,
`(m11, 2021-01-01 00:00:45)`.
However, since its timestamp precedes the malicious message `m'`,
it gets placed before `m'` in the list i.e.,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30)`.
In fact, for the next 10 hours,
`m'` will always be considered as the most recent message and
served as the last message to the querying nodes irrespective of how many other messages arrive afterward.
However, the lack of timestamp verifiability can create room for a range of attacks,
including injecting messages with invalid timestamps pointing to the far future.
To better understand the attack,
consider a store node whose current clock shows `2021-01-01 00:00:30` (and assume all the other nodes have a synchronized clocks +-20seconds).
The store node already has a list of messages,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20)`,
that are sorted based on their timestamp.
An attacker sends a message with an arbitrary large timestamp e.g.,
10 hours ahead of the correct clock `(m',2021-01-01 10:00:30)`.
The store node places `m'` at the end of the list,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m',2021-01-01 10:00:30)`.
Now another message arrives with a valid timestamp e.g.,
`(m11, 2021-01-01 00:00:45)`.
However, since its timestamp precedes the malicious message `m'`,
it gets placed before `m'` in the list i.e.,
`(m1,2021-01-01 00:00:00), (m2,2021-01-01 00:00:01), ..., (m10:2021-01-01 00:00:20), (m11, 2021-01-01 00:00:45), (m',2021-01-01 10:00:30)`.
In fact, for the next 10 hours,
`m'` will always be considered as the most recent message and
served as the last message to the querying nodes irrespective of how many other messages arrive afterward.
A robust and verifiable timestamp allows the receiver of a message to verify that a message has been generated prior to the claimed timestamp.
A robust and verifiable timestamp allows the receiver of a message to verify that a message has been generated prior to the claimed timestamp.
One solution is the use of [open timestamps](https://opentimestamps.org/) e.g.,
block height in Blockchain-based timestamps.
That is, messages contain the most recent block height perceived by their senders at the time of message generation.
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
seconds (e.g., in Ethereum 2.0) from the time of origination.
block height in Blockchain-based timestamps.
That is, messages contain the most recent block height perceived by their senders at the time of message generation.
This proves accuracy within a range of minutes (e.g., in Bitcoin blockchain) or
seconds (e.g., in Ethereum 2.0) from the time of origination.
# Copyright
@ -265,8 +386,8 @@ Copyright and related rights waived via
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
# References
1. [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14)
1. [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md)
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
3. [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/)
4. [Open timestamps](https://opentimestamps.org/)
3. [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/11/relay.md)
4. [Open timestamps](https://opentimestamps.org/)
5. [13/WAKU2-STORE v2 previous version](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/13/store.md)