mirror of https://github.com/vacp2p/rfc.git
adds pagination specs
adds some edits fixes minor typo WIP: integrates the pagination into store protocol updates wordlist.txt WIP: minor updates updates wordlist minor updates updates TOC updates the specs fixes a typo beta1 to beta2 deletes waku-store-pagination.md updates TOC with an autogenerated markdown-toc Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> itemizes the message fields of HistoryRPC some updates Update specs/waku/v2/waku-store.md Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com> WIP: initial reorganization WIP: shortens the pagination spec removes some comments and adds more details to the wire spec minor updates beta1 to beta2 converts direction to enum edits the Index description Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> shortens the abstract Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> Update specs/waku/v2/waku-store.md Co-authored-by: Oskar Thorén <ot@oskarthoren.com> edits changelog Update specs/waku/v2/waku-store.md Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com> Update specs/waku/v2/waku-store.md Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com> Update specs/waku/v2/waku-store.md Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com> Update specs/waku/v2/waku-store.md Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com> minor minor reformats the change log Update specs/waku/v2/waku-store.md Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com> edits version numbers updates the release date removes Table of content from Table of content! Update specs/waku/v2/waku-store.md Co-authored-by: Kim De Mey <kim.demey@gmail.com> edits the wordlist explains the case where pageSize is zero Update waku-store.md Update waku-v2.md adds an overview of the sorting approach minor Update waku-store.md
This commit is contained in:
Sanaz
parent
e7ec3fe0fa
commit
edc90625ff
|
|
@ -1,37 +1,59 @@
|
|||
---
|
||||
title: Waku
|
||||
version: 2.0.0-beta1
|
||||
version: 2.0.0-beta2
|
||||
status: Draft
|
||||
authors: Oskar Thorén <oskar@status.im>, Dean Eigenmann <dean@status.im>
|
||||
authors: Oskar Thorén <oskar@status.im>, Dean Eigenmann <dean@status.im>, Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
# Table of Contents
|
||||
|
||||
- [Abstract](#abstract)
|
||||
- [Wire Specification](#wire-specification)
|
||||
* [Protobuf](#protobuf)
|
||||
- [Protobuf](#protobuf)
|
||||
- [Index](#index)
|
||||
- [PagingInfo](#paginginfo)
|
||||
- [HistoryQuery](#historyquery)
|
||||
- [HistoryResponse](#historyresponse)
|
||||
- [Changelog](#changelog)
|
||||
- [2.0.0-beta2](#200-beta2)
|
||||
- [2.0.0-beta1](#200-beta1)
|
||||
- [Copyright](#copyright)
|
||||
|
||||
# Abstract
|
||||
|
||||
`WakuStore` is a protocol to enable querying of messages received through relay protocol and stored by other nodes.
|
||||
This specification explains the Waku Store protocol which enables querying of messages received through relay protocol and stored by other nodes. It also supports pagination for more efficient querying of historical messages.
|
||||
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta1`
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta2`
|
||||
|
||||
# Wire Specification
|
||||
|
||||
Peers communicate with each other using a request / response API. The messages sent are Protobuf RPC messages.
|
||||
Peers communicate with each other using a request / response API. The messages sent are Protobuf RPC messages. The followings are the specifications of the Protobuf messages.
|
||||
|
||||
## Protobuf
|
||||
|
||||
```protobuf
|
||||
message Index {
|
||||
bytes digest = 1;
|
||||
float receivedTime = 2;
|
||||
}
|
||||
|
||||
message PagingInfo {
|
||||
int64 pageSize = 1;
|
||||
Index cursor = 2;
|
||||
enum Direction {
|
||||
FORWARD = 0;
|
||||
BACKWARD = 1;
|
||||
}
|
||||
Direction direction = 3;
|
||||
}
|
||||
|
||||
message HistoryQuery {
|
||||
repeated string topics = 2;
|
||||
optional PagingInfo pagingInfo = 3; // used for pagination
|
||||
}
|
||||
|
||||
message HistoryResponse {
|
||||
repeated WakuMessage messages = 2;
|
||||
optional PagingInfo pagingInfo = 3; // used for pagination
|
||||
}
|
||||
|
||||
message HistoryRPC {
|
||||
|
|
@ -41,30 +63,44 @@ message HistoryRPC {
|
|||
}
|
||||
```
|
||||
|
||||
##### HistoryRPC
|
||||
### Index
|
||||
|
||||
A node MUST send all History messages (`HistoryQuery`, `HistoryResponse`) wrapped inside a
|
||||
`HistoryRPC`.
|
||||
To perform pagination, each `WakuMessage` stored at a node running the store protocol is associated with a unique `Index` that encapsulates the following parts.
|
||||
- `digest`: a sequence of bytes representing the hash of a `WakuMessage`.
|
||||
- `receivedTime`: the UNIX time at which the waku message is received by the node running the store protocol.
|
||||
|
||||
The `request_id` MUST be a uniquely generated string, the `HistoryResponse` and `HistoryQuery` `request_id` MUST match. The `HistoryResponse` message SHOULD contain any messages found
|
||||
whose `topic` can be found in the `HistoryQuery` `topics` array. Any message whose `topic` does not match MUST NOT be included.
|
||||
### PagingInfo
|
||||
|
||||
##### HistoryQuery
|
||||
`PagingInfo` holds the information required for pagination. It consists of the following components.
|
||||
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery` (or retrieved `WakuMessage`s in a `HistoryResponse`).
|
||||
- `cursor`: holds the `Index` of a `WakuMessage`.
|
||||
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
|
||||
|
||||
### HistoryQuery
|
||||
|
||||
RPC call to query historical messages.
|
||||
|
||||
The `topics` field MUST indicate the list of topics to query.
|
||||
- The `topics` field MUST indicate the list of topics to query.
|
||||
- `PagingInfo` holds the information required for pagination. Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`. If the `pageSize` is zero then no pagination is required. If the `pageSize` exceeds a threshold then the threshold value shall be used instead. In the forward pagination request, the `messages` field of the `HistoryResponse` shall contain at maximum the `pageSize` amount of waku messages whose `Index` values are larger than the given `cursor` (and vise versa for the backward pagination). Note that the `cursor` of a `HistoryQuery` may be empty (e.g., for the initial query), as such, and depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` waku messages shall be returned, respectively.
|
||||
The queried node MAY sort the `WakuMessage`s based on their `Index`, where the `receivedTime` constitutes the most significant part and the `digest` comes next, and then perform pagination on the sorted result. As such, the retrieved page contains an ordered list of `WakuMessage`s from the oldest message to the most recent one.
|
||||
|
||||
##### HistoryResponse
|
||||
### HistoryResponse
|
||||
|
||||
RPC call to respond to a HistoryQuery call.
|
||||
- The `messages` field MUST contain the messages found, these are [`WakuMessage`] types as defined in the corresponding [specification](./waku-message.md).
|
||||
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries. The `pageSize` indicates the number of returned waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`). The `direction` is the same direction as in the corresponding `HistoryQuery`. In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging). The requester shall embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the waku messages. The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result MAY be different.
|
||||
|
||||
The `messages` field MUST contain the messages found, these are [`WakuMessage`] types as defined in the corresponding [specification](./waku-message.md).
|
||||
|
||||
# Changelog
|
||||
|
||||
2.0.0-beta1
|
||||
Initial draft version. Released [2020-10-06](https://github.com/vacp2p/specs/commit/75b4c39e7945eb71ad3f9a0a62b99cff5dac42cf)
|
||||
|
||||
### 2.0.0-beta2
|
||||
Released [2020-10-27]()
|
||||
- Added pagination support.
|
||||
|
||||
### 2.0.0-beta1
|
||||
Released [2020-10-06](https://github.com/vacp2p/specs/commit/75b4c39e7945eb71ad3f9a0a62b99cff5dac42cf)
|
||||
- Initial draft version.
|
||||
|
||||
# Copyright
|
||||
|
||||
|
|
|
|||
|
|
@ -109,6 +109,8 @@ nim
|
|||
noop
|
||||
Ok
|
||||
Oskar
|
||||
pageSize
|
||||
pagingInfo
|
||||
peerid
|
||||
peerID
|
||||
Piana
|
||||
|
|
@ -116,12 +118,15 @@ Pluggable
|
|||
PoW
|
||||
proto
|
||||
protobuf
|
||||
PRs
|
||||
PSS
|
||||
pseudonymity
|
||||
PubSub
|
||||
pubsub
|
||||
pyspelling
|
||||
qNAN
|
||||
rebase
|
||||
receivedTime
|
||||
remoteHash
|
||||
remotelog
|
||||
RemoteLog
|
||||
|
|
@ -137,6 +142,7 @@ rlp
|
|||
rlpx
|
||||
RLPx
|
||||
rpc
|
||||
Sanaz
|
||||
scalability
|
||||
SECP
|
||||
semver
|
||||
|
|
@ -149,6 +155,7 @@ suboptimal
|
|||
SubOpts
|
||||
subprotocol
|
||||
subprotocols
|
||||
Taheri
|
||||
TBD
|
||||
TCP
|
||||
textlint
|
||||
|
|
|
|||
Loading…
Reference in New Issue