mirror of
https://github.com/logos-messaging/specs.git
synced 2026-01-05 23:53:12 +00:00
Merge branch 'master' into remove-meta
This commit is contained in:
commit
9abe5a850f
@ -27,9 +27,9 @@ This repository contains specifications for the Waku suite of protocols.
|
||||
|[WAKU2-ENR](standards/core/enr.md)| Waku Usage of ENR |
|
||||
|[WAKU2-INCENTIVIZATION](standards/core/incentivization.md)| Waku Incentivization |
|
||||
|[66/WAKU2-METADATA](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/66/metadata.md)| Waku Metadata |
|
||||
|[WAKU2-NETWORK](standards/core/network.md)| Waku Network |
|
||||
|[64/WAKU2-NETWORK](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md)| Waku Network |
|
||||
|[RELAY-SHARDING](standards/core/relay-sharding.md)| Waku Relay Sharding |
|
||||
| WAKU2-STOREV3 | Coming Soon |
|
||||
|[WAKU2-STORE](standards/core/store.md) | Waku Store Query |
|
||||
|
||||
### Application standards
|
||||
|
||||
|
||||
@ -26,7 +26,7 @@ this document lists static shard index assignments (see [WAKU2-RELAY-SHARDING](.
|
||||
| index | Protocol/App | Description |
|
||||
| ----- | ------------ | --------------------------------------------------------------- |
|
||||
| 0 | global | global use |
|
||||
| 1 | reserved | [The Waku Network](https://rfc.vac.dev/spec/64/#network-shards) |
|
||||
| 1 | reserved | [The Waku Network](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md) |
|
||||
| 2 | reserved | |
|
||||
| 3 | reserved | |
|
||||
| 4 | reserved | |
|
||||
@ -53,3 +53,4 @@ Copyright and related rights waived via [CC0](https://creativecommons.org/public
|
||||
|
||||
- [WAKU2-RELAY-SHARDING](../standards/core/relay-sharding.md)
|
||||
- [IANA port allocation](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml)
|
||||
- [The Waku Network](https://github.com/vacp2p/rfc-index/blob/main/waku/standards/core/64/network.md)
|
||||
|
||||
@ -1,51 +1,51 @@
|
||||
---
|
||||
title: 13/WAKU2-STORE
|
||||
name: Waku v2 Store
|
||||
status: draft
|
||||
tags: [waku-core]
|
||||
editor: Sanaz Taheri <sanaz@status.im>
|
||||
title: WAKU2-STORE
|
||||
name: Waku Store Query
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskarth@titanproxy.com>
|
||||
- Aaryamann Challani <aaryamann@status.im>
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
> **Note:** This version of WAKU2-STORE is earmarked to replace RFC [13/WAKU2-STORE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/13/store.md) once it reaches draft status
|
||||
|
||||
---
|
||||
|
||||
# Abstract
|
||||
|
||||
This specification explains the `13/WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
|
||||
This specification explains the `WAKU2-STORE` protocol which enables querying of messages received through the relay protocol and
|
||||
stored by other nodes.
|
||||
It also supports pagination for more efficient querying of historical messages.
|
||||
|
||||
**Protocol identifier\***: `/vac/waku/store/2.0.0-beta4`
|
||||
**Protocol identifier***: `/vac/waku/store-query/3.0.0`
|
||||
|
||||
## Terminology
|
||||
|
||||
The term PII, Personally Identifiable Information,
|
||||
refers to any piece of data that can be used to uniquely identify a user.
|
||||
For example, the signature verification key, and
|
||||
the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
# Design Requirements
|
||||
|
||||
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
|
||||
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
Nodes willing to provide the storage service using `13/WAKU2-STORE` protocol,
|
||||
Nodes willing to provide the storage service using `WAKU2-STORE` protocol,
|
||||
SHOULD provide a complete and full view of message history.
|
||||
As such, they are required to be _highly available_ and
|
||||
specifically have a _high uptime_ to consistently receive and store network messages.
|
||||
As such, they are required to be *highly available* and
|
||||
specifically have a *high uptime* to consistently receive and store network messages.
|
||||
The high uptime requirement makes sure that no message is missed out hence a complete and
|
||||
intact view of the message history is delivered to the querying nodes.
|
||||
Nevertheless, in case storage provider nodes cannot afford high availability,
|
||||
the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
|
||||
|
||||
The concept of `ephemeral` messages introduced in [`14/WAKU2-MESSAGE`](https://rfc.vac.dev/spec/14) affects `13/WAKU2-STORE` as well.
|
||||
Nodes running `13/WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
|
||||
Nodes running `13/WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
|
||||
The concept of `ephemeral` messages introduced in [`WAKU2-MESSAGE`](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) affects `WAKU2-STORE` as well.
|
||||
Nodes running `WAKU2-STORE` SHOULD support `ephemeral` messages as specified in [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md).
|
||||
Nodes running `WAKU2-STORE` SHOULD NOT store messages with the `ephemeral` flag set to `true`.
|
||||
|
||||
# Adversarial Model
|
||||
|
||||
Any peer running the `13/WAKU2-STORE` protocol, i.e.
|
||||
Any peer running the `WAKU2-STORE` protocol, i.e.
|
||||
both the querying node and the queried node, are considered as an adversary.
|
||||
Furthermore,
|
||||
we currently consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but
|
||||
@ -54,155 +54,277 @@ As we evolve the protocol,
|
||||
further adversarial models will be considered.
|
||||
For example, under the passive adversarial model,
|
||||
no malicious node hides or
|
||||
lies about the history of messages as it is against the description of the `13/WAKU2-STORE` protocol.
|
||||
lies about the history of messages as it is against the description of the `WAKU2-STORE` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication).
|
||||
In specific, the communication channels are assumed to be secure.
|
||||
|
||||
# Wire Specification
|
||||
|
||||
Peers communicate with each other using a request / response API.
|
||||
The messages sent are Protobuf RPC messages which are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
The following are the specifications of the Protobuf messages.
|
||||
|
||||
## Payloads
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
message Index {
|
||||
bytes digest = 1;
|
||||
sint64 receiverTime = 2;
|
||||
sint64 senderTime = 3;
|
||||
string pubsubTopic = 4;
|
||||
// Protocol identifier: /vac/waku/store-query/3.0.0
|
||||
package waku.store.v3;
|
||||
|
||||
import "waku/message/v1/message.proto";
|
||||
|
||||
message WakuMessageKeyValue {
|
||||
optional bytes message_hash = 1; // Globally unique key for a Waku Message
|
||||
|
||||
// Full message content and associated pubsub_topic as value
|
||||
optional waku.message.v1.WakuMessage message = 2;
|
||||
optional string pubsub_topic = 3;
|
||||
}
|
||||
|
||||
message PagingInfo {
|
||||
uint64 pageSize = 1;
|
||||
Index cursor = 2;
|
||||
enum Direction {
|
||||
BACKWARD = 0;
|
||||
FORWARD = 1;
|
||||
}
|
||||
Direction direction = 3;
|
||||
}
|
||||
|
||||
message ContentFilter {
|
||||
string contentTopic = 1;
|
||||
}
|
||||
|
||||
message HistoryQuery {
|
||||
// the first field is reserved for future use
|
||||
string pubsubtopic = 2;
|
||||
repeated ContentFilter contentFilters = 3;
|
||||
PagingInfo pagingInfo = 4;
|
||||
}
|
||||
|
||||
message HistoryResponse {
|
||||
// the first field is reserved for future use
|
||||
repeated WakuMessage messages = 2;
|
||||
PagingInfo pagingInfo = 3;
|
||||
enum Error {
|
||||
NONE = 0;
|
||||
INVALID_CURSOR = 1;
|
||||
}
|
||||
Error error = 4;
|
||||
}
|
||||
|
||||
message HistoryRPC {
|
||||
message StoreQueryRequest {
|
||||
string request_id = 1;
|
||||
HistoryQuery query = 2;
|
||||
HistoryResponse response = 3;
|
||||
bool include_data = 2; // Response should include full message content
|
||||
|
||||
// Filter criteria for content-filtered queries
|
||||
optional string pubsub_topic = 10;
|
||||
repeated string content_topics = 11;
|
||||
optional sint64 time_start = 12;
|
||||
optional sint64 time_end = 13;
|
||||
|
||||
// List of key criteria for lookup queries
|
||||
repeated bytes message_hashes = 20; // Message hashes (keys) to lookup
|
||||
|
||||
// Pagination info. 50 Reserved
|
||||
optional bytes pagination_cursor = 51; // Message hash (key) from where to start query (exclusive)
|
||||
bool pagination_forward = 52;
|
||||
optional uint64 pagination_limit = 53;
|
||||
}
|
||||
|
||||
message StoreQueryResponse {
|
||||
string request_id = 1;
|
||||
|
||||
optional uint32 status_code = 10;
|
||||
optional string status_desc = 11;
|
||||
|
||||
repeated WakuMessageKeyValue messages = 20;
|
||||
|
||||
optional bytes pagination_cursor = 51;
|
||||
}
|
||||
```
|
||||
## General store query concepts
|
||||
|
||||
### Index
|
||||
### Waku message key-value pairs
|
||||
|
||||
To perform pagination,
|
||||
each `WakuMessage` stored at a node running the `13/WAKU2-STORE` protocol is associated with a unique `Index` that encapsulates the following parts.
|
||||
The store query protocol operates as a query protocol for a key-value store of historical Waku messages,
|
||||
with each entry having a [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) and associated pubsub topic as value,
|
||||
and [deterministic message hash](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md#deterministic-message-hashing) as key.
|
||||
The store can be queried to return either a set of keys or a set of key-value pairs.
|
||||
Within the store query protocol, Waku message keys and values MUST be represented in a `WakuMessageKeyValue` message.
|
||||
This message MUST contain the deterministic `message_hash` as key.
|
||||
It MAY contain the full `WakuMessage` and associated pubsub topic as value in the `message` and `pubsub_topic` fields,
|
||||
depending on the use case as set out below.
|
||||
If the message contains a value entry in addition to the key,
|
||||
both the `message` and `pubsub_topic` fields MUST be populated.
|
||||
The message MUST NOT have either `message` or `pubsub_topic` populated with the other unset.
|
||||
Both fields MUST either be set or unset.
|
||||
|
||||
- `digest`: a sequence of bytes representing the SHA256 hash of a `WakuMessage`.
|
||||
The hash is computed over the concatenation of `contentTopic` and `payload` fields of a `WakuMessage` (see [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14)).
|
||||
- `receiverTime`: the UNIX time in nanoseconds at which the `WakuMessage` is received by the receiving node.
|
||||
- `senderTime`: the UNIX time in nanoseconds at which the `WakuMessage` is generated by its sender.
|
||||
- `pubsubTopic`: the pubsub topic on which the `WakuMessage` is received.
|
||||
### Waku message store eligibility
|
||||
|
||||
### PagingInfo
|
||||
In order for a Waku message to be eligible for storage:
|
||||
- it MUST be a _valid_ [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md).
|
||||
- the `timestamp` field MUST be populated with the Unix epoch time at which the message was generated in nanoseconds.
|
||||
If at the time of storage the `timestamp` deviates by more than 20 seconds
|
||||
either into the past or the future when compared to the store node’s internal clock,
|
||||
the store node MAY reject the message.
|
||||
- the `ephemeral` field MUST be set to `false`.
|
||||
|
||||
`PagingInfo` holds the information required for pagination. It consists of the following components.
|
||||
### Waku message sorting
|
||||
|
||||
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery`
|
||||
(or retrieved `WakuMessage`s in a `HistoryResponse`).
|
||||
- `cursor`: holds the `Index` of a `WakuMessage`.
|
||||
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
|
||||
The key-value entries in the store MUST be time-sorted by the `WakuMessage` `timestamp` attribute.
|
||||
Where two or more key-value entries have identical `timestamps`,
|
||||
the entries MUST be further sorted by the natural order of their message hash keys.
|
||||
Within the context of traversing over key-value entries in the store,
|
||||
_"forward"_ indicates traversing the entries in ascending order,
|
||||
whereas _"backward"_ indicates traversing the entries in descending order.
|
||||
|
||||
### ContentFilter
|
||||
### Pagination
|
||||
|
||||
`ContentFilter` carries the information required for filtering historical messages.
|
||||
If a large number of entries in the store service node match the query criteria provided in a `StoreQueryRequest`,
|
||||
the client MAY make use of pagination
|
||||
in a chain of store query request and response transactions
|
||||
to retrieve the full response in smaller batches termed _"pages"_.
|
||||
Pagination can be performed either in [a _forward_ or _backward_ direction](#waku-message-sorting).
|
||||
|
||||
- `contentTopic` represents the content topic of the queried historical `WakuMessage`.
|
||||
This field maps to the `contentTopic` field of the [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
|
||||
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
|
||||
by setting the page size limit in the `pagination_limit` field.
|
||||
Note that a store service node MAY enforce its own limit
|
||||
if the `pagination_limit` is unset
|
||||
or larger than the service node's internal page size limit.
|
||||
|
||||
### HistoryQuery
|
||||
A `StoreQueryResponse` with a populated `pagination_cursor` indicates that more stored entries match the query than included in the response.
|
||||
|
||||
RPC call to query historical messages.
|
||||
A `StoreQueryResponse` without a populated `pagination_cursor` indicates that
|
||||
there are no more matching entries in the store.
|
||||
|
||||
- The `pubsubTopic` field MUST indicate the pubsub topic of the historical messages to be retrieved.
|
||||
This field denotes the pubsub topic on which `WakuMessage`s are published.
|
||||
This field maps to `topicIDs` field of `Message` in [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/).
|
||||
Leaving this field empty means no filter on the pubsub topic of message history is requested.
|
||||
This field SHOULD be left empty in order to retrieve the historical `WakuMessage` regardless of the pubsub topics on which they are published.
|
||||
- The `contentFilters` field MUST indicate the list of content filters based on which the historical messages are to be retrieved.
|
||||
Leaving this field empty means no filter on the content topic of message history is required.
|
||||
This field SHOULD be left empty in order to retrieve historical `WakuMessage` regardless of their content topics.
|
||||
- `PagingInfo` holds the information required for pagination.
|
||||
Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`.
|
||||
It is RECOMMENDED that the queried node defines a maximum page size internally.
|
||||
If the querying node leaves the `pageSize` unspecified,
|
||||
or if the `pageSize` exceeds the maximum page size,
|
||||
the queried node SHOULD auto-paginate the `HistoryResponse` to no more than the configured maximum page size.
|
||||
This allows mitigation of long response time for `HistoryQuery`.
|
||||
In the forward pagination request,
|
||||
the `messages` field of the `HistoryResponse` SHALL contain, at maximum,
|
||||
the `pageSize` amount of `WakuMessage` whose `Index` values are larger than the given `cursor`
|
||||
(and vise versa for the backward pagination).
|
||||
Note that the `cursor` of a `HistoryQuery` MAY be empty (e.g., for the initial query), as such, and
|
||||
depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` `WakuMessage` SHALL be returned, respectively.
|
||||
The client MAY request the next page of entries from the store service node
|
||||
by populating a subsequent `StoreQueryRequest` with the `pagination_cursor` received in the `StoreQueryResponse`.
|
||||
All other fields and query criteria MUST be the same as in the preceding `StoreQueryRequest`.
|
||||
|
||||
### Sorting Messages
|
||||
A `StoreQueryRequest` without a populated `pagination_cursor` indicates that
|
||||
the client wants to retrieve the "first page" of the stored entries matching the query.
|
||||
|
||||
The queried node MUST sort the `WakuMessage` based on their `Index`,
|
||||
where the `senderTime` constitutes the most significant part and the `digest` comes next, and
|
||||
then perform pagination on the sorted result.
|
||||
As such, the retrieved page contains an ordered list of `WakuMessage` from the oldest messages to the most recent one.
|
||||
Alternatively, the `receiverTime` (instead of `senderTime` ) MAY be used to sort messages during the paging process.
|
||||
However, it is RECOMMENDED the use of the `senderTime` for sorting as it is invariant and
|
||||
consistent across all the nodes.
|
||||
This has the benefit of `cursor` reusability i.e.,
|
||||
a `cursor` obtained from one node can be consistently used to query from another node.
|
||||
However, this `cursor` reusability does not hold when the `receiverTime` is utilized as the receiver time is affected by the network delay and
|
||||
nodes' clock asynchrony.
|
||||
## Store Query Request
|
||||
|
||||
### HistoryResponse
|
||||
A client node MUST send all historical message queries within a `StoreQueryRequest` message.
|
||||
This request MUST contain a `request_id`.
|
||||
The `request_id` MUST be a uniquely generated string.
|
||||
|
||||
RPC call to respond to a HistoryQuery call.
|
||||
If the store query client requires the store service node to include Waku message values in the query response,
|
||||
it MUST set `include_data` to `true`.
|
||||
If the store query client requires the store service node to return only message hash keys in the query response,
|
||||
it SHOULD set `include_data` to `false`.
|
||||
By default, therefore, the store service node assumes `include_data` to be `false`.
|
||||
|
||||
- The `messages` field MUST contain the messages found,
|
||||
these are [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14) types.
|
||||
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries.
|
||||
The `pageSize` indicates the number of returned Waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`).
|
||||
The `direction` is the same direction as in the corresponding `HistoryQuery`.
|
||||
In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging).
|
||||
Regardless of the paging direction, the retrieved `messages` are always sorted in ascending order based on their timestamp as explained in the [sorting messages](#sorting-messages) section, that is, from the oldest to the most recent.
|
||||
The requester SHALL embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
|
||||
The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result may be different.
|
||||
- The `error` field contains information about any error that has occurred while processing the corresponding `HistoryQuery`.
|
||||
`NONE` stands for no error.
|
||||
This is also the default value.
|
||||
`INVALID_CURSOR` means that the `cursor` field of `HistoryQuery` does not match with the `Index` of any of the `WakuMessage` persisted by the queried node.
|
||||
A store query client MAY include query filter criteria in the `StoreQueryRequest`.
|
||||
There are two types of filter use cases:
|
||||
1. Content filtered queries and
|
||||
2. Message hash lookup queries
|
||||
|
||||
### Content filtered queries
|
||||
|
||||
A store query client MAY request the store service node to filter historical entries by a content filter.
|
||||
Such a client MAY create a filter on content topic, on time range or on both.
|
||||
|
||||
To filter on content topic, the client MUST populate _both_ the `pubsub_topic` _and_ `content_topics` field.
|
||||
The client MUST NOT populate either `pubsub_topic` or `content_topics` and leave the other unset.
|
||||
Both fields MUST either be set or unset.
|
||||
A mixed content topic filter with just one of either `pubsub_topic` or `content_topics` set, SHOULD be regarded as an invalid request.
|
||||
|
||||
To filter on time range, the client MUST set `time_start`, `time_end` or both.
|
||||
Each `time_` field should contain a Unix epoch timestamp in nanoseconds.
|
||||
An unset `time_start` SHOULD be interpreted as "from the oldest stored entry".
|
||||
An unset `time_end` SHOULD be interpreted as "up to the youngest stored entry".
|
||||
|
||||
If any of the content filter fields are set,
|
||||
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`,
|
||||
the client MUST NOT set the `message_hashes` field.
|
||||
|
||||
### Message hash lookup queries
|
||||
|
||||
A store query client MAY request the store service node to filter historical entries by one or more matching message hash keys.
|
||||
This type of query acts as a "lookup" against a message hash key or set of keys already known to the client.
|
||||
|
||||
In order to perform a lookup query, the store query client MUST populate the `message_hashes` field with the list of message hash keys it wants to lookup in the store service node.
|
||||
|
||||
If the `message_hashes` field is set,
|
||||
the client MUST NOT set any of the content filter fields,
|
||||
namely `pubsub_topic`, `content_topic`, `time_start`, or `time_end`.
|
||||
|
||||
### Presence queries
|
||||
|
||||
A presence query is a special type of lookup query that allows a client to check for the presence of one or more messages in the store service node,
|
||||
without retrieving the full contents (values) of the messages.
|
||||
This can, for example, be used as part of a reliability mechanism,
|
||||
whereby store query clients verify that previously published messages have been successfully stored.
|
||||
|
||||
In order to perform a presence query,
|
||||
the store query client MUST populate the `message_hashes` field in the `StoreQueryRequest` with the list of message hashes
|
||||
for which it wants to verify presence in the store service node.
|
||||
The `include_data` property MUST be set to `false`.
|
||||
The client SHOULD interpret every `message_hash` returned in the `messages` field of the `StoreQueryResponse` as present in the store.
|
||||
The client SHOULD assume that all other message hashes included in the original `StoreQueryRequest` but not in the `StoreQueryResponse` is not present in the store.
|
||||
|
||||
### Pagination info
|
||||
|
||||
The store query client MAY include a message hash as `pagination_cursor`,
|
||||
to indicate at which key-value entry a store service node SHOULD start the query.
|
||||
The `pagination_cursor` is treated as exclusive
|
||||
and the corresponding entry will not be included in subsequent store query responses.
|
||||
|
||||
For forward queries, only messages following (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor` will be returned.
|
||||
For backward queries, only messages preceding (see [sorting](#waku-message-sorting)) the one indexed at `pagination_cursor` will be returned.
|
||||
|
||||
If the store query client requires the store service node to perform a forward query,
|
||||
it MUST set `pagination_forward` to `true`.
|
||||
If the store query client requires the store service node to perform a backward query,
|
||||
it SHOULD set `pagination_forward` to `false`.
|
||||
By default, therefore, the store service node assumes pagination to be backward.
|
||||
|
||||
A store query client MAY indicate the maximum number of matching entries it wants in the `StoreQueryResponse`,
|
||||
by setting the page size limit in the `pagination_limit` field.
|
||||
Note that a store service node MAY enforce its own limit
|
||||
if the `pagination_limit` is unset
|
||||
or larger than the service node's internal page size limit.
|
||||
|
||||
See [pagination](#pagination) for more on how the pagination info is used in store transactions.
|
||||
|
||||
## Store Query Response
|
||||
|
||||
In response to any `StoreQueryRequest`,
|
||||
a store service node SHOULD respond with a `StoreQueryResponse` with a `requestId` matching that of the request.
|
||||
This response MUST contain a `status_code` indicating if the request was successful or not.
|
||||
Successful status codes are in the `2xx` range.
|
||||
Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
|
||||
In addition, the store service node MAY choose to provide a more detailed status description in the `status_desc` field.
|
||||
|
||||
### Filter matching
|
||||
|
||||
For [content filtered queries](#content-filtered-queries), an entry in the store service node matches the filter criteria in a `StoreQueryRequest` if each of the following conditions are met:
|
||||
- its `content_topic` is in the request `content_topics` set
|
||||
and it was published on a matching `pubsub_topic` OR the request `content_topics` and `pubsub_topic` fields are unset
|
||||
- its `timestamp` is _larger or equal_ than the request `start_time` OR the request `start_time` is unset
|
||||
- its `timestamp` is _smaller_ than the request `end_time` OR the request `end_time` is unset
|
||||
|
||||
Note that for content filtered queries, `start_time` is treated as _inclusive_ and `end_time` is treated as _exclusive_.
|
||||
|
||||
For [message hash lookup queries](#message-hash-lookup-queries), an entry in the store service node matches the filter criteria if its `message_hash` is in the request `message_hashes` set.
|
||||
|
||||
The store service node SHOULD respond with an error code and discard the request
|
||||
if the store query request contains both content filter criteria and message hashes.
|
||||
|
||||
### Populating response messages
|
||||
|
||||
The store service node SHOULD populate the `messages` field in the response
|
||||
only with entries matching the filter criteria provided in the corresponding request.
|
||||
Regardless of whether the response is to a _forward_ or _backward_ query,
|
||||
the `messages`field in the response MUST be ordered in a forward direction
|
||||
according to the [message sorting rules](#waku-message-sorting).
|
||||
|
||||
If the corresponding `StoreQueryRequest` has `include_data` set to true,
|
||||
the service node SHOULD populate both the `message_hash` and `message` for each entry in the response.
|
||||
In all other cases, the store service node SHOULD populate only the `message_hash` field for each entry in the response.
|
||||
|
||||
### Paginating the response
|
||||
|
||||
The response SHOULD NOT contain more `messages` than the `pagination_limit` provided in the corresponding `StoreQueryRequest`.
|
||||
It is RECOMMENDED that the store node defines its own maximum page size internally.
|
||||
If the `pagination_limit` in the request is unset,
|
||||
or exceeds this internal maximum page size,
|
||||
the store service node SHOULD ignore the `pagination_limit` field and apply its own internal maximum page size.
|
||||
|
||||
In response to a _forward_ `StoreQueryRequest`:
|
||||
- if the `pagination_cursor` is set,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries following the `pagination_cursor` (exclusive).
|
||||
- if the `pagination_cursor` is unset,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries from the first entry in the store.
|
||||
- if there are still more matching entries in the store
|
||||
after the maximum page size is reached while populating the response,
|
||||
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
|
||||
with the message hash key of the _last_ entry _included_ in the response.
|
||||
|
||||
In response to a _backward_ `StoreQueryRequest`:
|
||||
- if the `pagination_cursor` is set,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries preceding the `pagination_cursor` (exclusive).
|
||||
- if the `pagination_cursor` is unset,
|
||||
the store service node SHOULD populate the `messages` field
|
||||
with matching entries from the last entry in the store.
|
||||
- if there are still more matching entries in the store
|
||||
after the maximum page size is reached while populating the response,
|
||||
the store service node SHOULD populate the `pagination_cursor` in the `StoreQueryResponse`
|
||||
with the message hash key of the _first_ entry _included_ in the response.
|
||||
|
||||
# Security Consideration
|
||||
|
||||
@ -211,14 +333,13 @@ The main security consideration to take into account while using this protocol i
|
||||
# Future Work
|
||||
|
||||
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes i.e.,
|
||||
without disclosing the exact topics of [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14) they are interested in.
|
||||
As such, no adversary in the `13/WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
|
||||
content topics of [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14).
|
||||
The current version of the `13/WAKU2-STORE` protocol does not provide anonymity for historical queries,
|
||||
as the querying node needs to directly connect to another node in the `13/WAKU2-STORE` protocol and
|
||||
without disclosing the exact topics of [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md) they are interested in.
|
||||
As such, no adversary in the `WAKU2-STORE` protocol would be able to learn which peer is interested in which content filters i.e.,
|
||||
content topics of [14/WAKU2-MESSAGE](/spec/14).
|
||||
The current version of the `WAKU2-STORE` protocol does not provide anonymity for historical queries,
|
||||
as the querying node needs to directly connect to another node in the `WAKU2-STORE` protocol and
|
||||
explicitly disclose the content filters of its interest to retrieve the corresponding messages.
|
||||
However, one can consider preserving anonymity through one of the following ways:
|
||||
|
||||
- By hiding the source of the request i.e., anonymous communication.
|
||||
That is the querying node shall hide all its PII in its history request e.g., its IP address.
|
||||
This can happen by the utilization of a proxy server or by using Tor.
|
||||
@ -265,8 +386,8 @@ Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
# References
|
||||
|
||||
1. [14/WAKU2-MESSAGE](https://rfc.vac.dev/spec/14)
|
||||
1. [14/WAKU2-MESSAGE](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/14/message.md)
|
||||
2. [protocol buffers v3](https://developers.google.com/protocol-buffers/)
|
||||
3. [11/WAKU2-RELAY](https://rfc.vac.dev/spec/11/)
|
||||
3. [11/WAKU2-RELAY](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/11/relay.md)
|
||||
4. [Open timestamps](https://opentimestamps.org/)
|
||||
5. [13/WAKU2-STORE v2 previous version](https://github.com/vacp2p/rfc-index/blob/7b443c1aab627894e3f22f5adfbb93f4c4eac4f6/waku/standards/core/13/store.md)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user