Merge repo with rfc.vac.dev (#310)
* Hugo init * Init Hugo book * Add basic README and title * Import RFC: 1/COSS * Move example content to dedicated folder * Basic menu * Tweak single layout * Fix base url and permalink * Basic index tweaks * Add netifly config * gitignore * 1/COSS: Add COSS lifecycle * Remove public artifact * Add basic numbers in menu * Empty READMEs init * Import mvds spec raw * 2/MVDS: Simplify * 3/REMOTE-LOG: Import * 3/REMOTE-LOG: Simplify * 4/MVDS-META: Import * 4/MVDS-META Simplify * Tweaks * 5/WAKU0: Import * 5: Simplify * 6/WAKU1: Import * 6/WAKU1: Simplify * 7/WAKU-DATA: Import * 7/WAKU-DATA: Simplify * 8/WAKU-MAIL: Import * 8/WAKU-MAIL: Simplify * Tweak * 9/WAKU-RPC: Import * 9/WAKU-RPC: Simplify * 10/WAKU2: Import * 10/WAKU2: Simplify * 11/WAKU-RELAY: Import * Tweaks * 11/WAKU-RELAY: Simplify * 12/WAKU-FILTER: Import * 12/WAKU2-FILTER: Simplify * 13/WAKU-STORE: Import * 13/WAKU-STORE: Simplify * 14/WAKU-MESSAGE: Import * 14/WAKU-MESSAGE: Simplify * 15/WAKU-BRIDGE: Import * 15: Simplify * 16: Import * 16: Simplify * 17: Import * 17: Simplify * 18: Import * 18: Simplify * 12: Bump update * 14: Bump update * 11: Bump update * 17: Bump update * 18: Bump update * 16: Bump update * Fix slug for 18 * Editor changes for 11, 12, 15 * Revert "Editor changes for 11, 12, 15" This reverts commit5c32fe791b
. * Revert "Revert "Editor changes for 11, 12, 15"" This reverts commitccd75131d7
. * Editor changes for 13, 2, 4 * Move current specs to archive * Update README * bump
|
@ -1 +1,6 @@
|
|||
node_modules/
|
||||
.DS_Store
|
||||
.idea
|
||||
*.log
|
||||
tmp/
|
||||
public
|
||||
|
|
|
@ -0,0 +1,3 @@
|
|||
[submodule "themes/book"]
|
||||
path = themes/book
|
||||
url = https://github.com/alex-shpak/hugo-book
|
79
README.md
|
@ -1,83 +1,30 @@
|
|||
<!-- [CI](https://github.com/vacp2p/specs/workflows/CI/badge.svg) -->
|
||||
# rfc.vac.dev
|
||||
|
||||
This repository contains the specs for [vac](https://vac.dev), a modular peer-to-peer messaging stack, with a focus on secure messaging. A detailed explanation of the vac and its design goals can be found [here](https://vac.dev/vac-overview).
|
||||
|
||||
## Status
|
||||
See [rfc.vac.dev](rfc.vac.dev).
|
||||
|
||||
The entire vac protocol is under active development, each specification has its own `status` which is reflected through the version number at the top of every document. We use [semver](https://semver.org/) to version these specifications.
|
||||
## Contributing
|
||||
|
||||
## Protocols
|
||||
Please see [1/COSS](https://rfc.vac.dev/spec/1/) for general guidelines and spec lifecycle.
|
||||
|
||||
### Building locally
|
||||
|
||||
Ensure you have the Hugo extended edition
|
||||
(https://gohugo.io/getting-started/installing/), then run `hugo server`.
|
||||
|
||||
These protocols define various components of the [vac](https://vac.dev) stack.
|
||||
|
||||
### Waku
|
||||
|
||||
Waku is a protocol that replaces Whisper ([EIP-627](https://eips.ethereum.org/EIPS/eip-627)). Waku v2 is an upgrade of Waku v1 that is in active development and offer significant improvements. You can read more about the [plan for it](https://vac.dev/waku-v2-plan) and an [update](https://vac.dev/waku-v2-update).
|
||||
|
||||
#### Version 2
|
||||
- [waku/2](./specs/waku/v2/waku-v2) specs for `waku/2` version, current draft version
|
||||
- [waku/2 relay](./specs/waku/v2/waku-relay) spec for WakuRelay, current draft version
|
||||
- [waku/2 filter](./specs/waku/v2/waku-filter) spec for WakuFilter, current draft version
|
||||
- [waku/2 store](./specs/waku/v2/waku-store) spec for WakuStore, current draft version
|
||||
- [waku/2 message](./specs/waku/v2/waku-message) spec for Waku Message, current draft version
|
||||
- [waku/2 bridge](./specs/waku/v2/waku-bridge) spec for Waku bridge with v1, alpha
|
||||
- [waku/2 rln relay](./specs/waku/v2/waku-rln-relay) spec for Waku Relay with RLN, alpha
|
||||
- [waku/2 swap](./specs/waku/v2/waku-swap-accounting) spec for Waku Swap Accounting, alpha
|
||||
- [waku/2 rpc api](./specs/waku/v2/waku-v2-rpc-api) - Waku RPC API for Waku v2 nodes, alpha
|
||||
|
||||
#### Version 0 and 1
|
||||
- [waku/0](./specs/waku/v1/waku-0) specs for `waku/0` version, now deprecated
|
||||
- [waku/1](./specs/waku/v1/waku-1) specs for `waku/1` version, current stable version
|
||||
- [envelope data format](./specs/waku/v1/envelope-data-format) [waku](./waku/waku) envelope data field specification.
|
||||
- [mailserver](./specs/waku/v1/mailserver) - Mailserver specification for archiving and delivering historical [waku](./specs/waku/waku) envelopes on demand.
|
||||
- [rpc api](./specs/waku/v1/waku-rpc-api) - Waku RPC API for Waku v1 nodes.
|
||||
|
||||
### Data sync
|
||||
|
||||
- [mvds](./specs/mvds) - Data Synchronization protocol for unreliable transports.
|
||||
- [remote log](./specs/remote-log) - Remote replication of local logs.
|
||||
- [mvds metadata](./specs/mvds-metadata) - Metadata field for [MVDS](./specs/mvds) messages.
|
||||
|
||||
## Style guide
|
||||
### Style guide
|
||||
|
||||
Sequence diagrams are generated using [Mscgen](http://www.mcternan.me.uk/mscgen/) like this: `mscgen -T png -i input.msc -o output.png`. Both the source and generated image should be in source control. For ease of readability, the generated image is embedded inside the main spec document.
|
||||
|
||||
Alternatively, [mscgenjs](https://github.com/mscgenjs/mscgenjs-cli) can be used to generate sequence diagrams (mscgenjs produces better quality figures especially concerning lines' spaces and figures' margins). Once installed, the following command can be used to generate the sequence diagrams `mscgenjs -T png -i input.msc -o output.png`. More details on the installation and compilation are given in [mscgenjs repository](https://github.com/mscgenjs/mscgenjs-cli). You may try the online playground https://mscgen.js.org/ as well to get a sense of the output figures.
|
||||
|
||||
The lifecycle of the specs follows the [COSS Lifecycle](https://rfc.unprotocols.org/spec:2/COSS/)
|
||||
|
||||
## Meta
|
||||
### Meta
|
||||
|
||||
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
## Linting
|
||||
## Acknowledgement
|
||||
|
||||
### Spellcheck
|
||||
|
||||
To run the spellchecker locally, you must install [pyspelling](https://facelessuser.github.io/pyspelling/).
|
||||
|
||||
It can then be run with the following command:
|
||||
|
||||
```console
|
||||
pyspelling -c .pyspelling.yml
|
||||
```
|
||||
|
||||
Words that should be ignored or are unrecognized must be added to the [wordlist](./wordlist.txt).
|
||||
|
||||
### Markdown Verification
|
||||
|
||||
We use [remark](https://remark.js.org/) to verify our markdown. You can easily run this tool simply by using our `npm` package:
|
||||
|
||||
```console
|
||||
npm install
|
||||
npm run lint
|
||||
```
|
||||
|
||||
### Textlint
|
||||
|
||||
We use [textlint](https://textlint.github.io/) for extra markdown verification. You can easily run this tool simply by using our `npm` package:
|
||||
|
||||
```console
|
||||
npm install
|
||||
npm run textlint
|
||||
```
|
||||
Site layout and process inspired by https://rfc.zeromq.org/
|
||||
|
|
|
@ -0,0 +1,6 @@
|
|||
---
|
||||
title: "{{ replace .Name "-" " " | title }}"
|
||||
date: {{ .Date }}
|
||||
draft: true
|
||||
---
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
<!-- [CI](https://github.com/vacp2p/specs/workflows/CI/badge.svg) -->
|
||||
|
||||
This repository contains the specs for [vac](https://vac.dev), a modular peer-to-peer messaging stack, with a focus on secure messaging. A detailed explanation of the vac and its design goals can be found [here](https://vac.dev/vac-overview).
|
||||
|
||||
## Status
|
||||
|
||||
The entire vac protocol is under active development, each specification has its own `status` which is reflected through the version number at the top of every document. We use [semver](https://semver.org/) to version these specifications.
|
||||
|
||||
## Protocols
|
||||
|
||||
These protocols define various components of the [vac](https://vac.dev) stack.
|
||||
|
||||
### Waku
|
||||
|
||||
Waku is a protocol that replaces Whisper ([EIP-627](https://eips.ethereum.org/EIPS/eip-627)). Waku v2 is an upgrade of Waku v1 that is in active development and offer significant improvements. You can read more about the [plan for it](https://vac.dev/waku-v2-plan) and an [update](https://vac.dev/waku-v2-update).
|
||||
|
||||
#### Version 2
|
||||
- [waku/2](./specs/waku/v2/waku-v2) specs for `waku/2` version, current draft version
|
||||
- [waku/2 relay](./specs/waku/v2/waku-relay) spec for WakuRelay, current draft version
|
||||
- [waku/2 filter](./specs/waku/v2/waku-filter) spec for WakuFilter, current draft version
|
||||
- [waku/2 store](./specs/waku/v2/waku-store) spec for WakuStore, current draft version
|
||||
- [waku/2 message](./specs/waku/v2/waku-message) spec for Waku Message, current draft version
|
||||
- [waku/2 bridge](./specs/waku/v2/waku-bridge) spec for Waku bridge with v1, alpha
|
||||
- [waku/2 rln relay](./specs/waku/v2/waku-rln-relay) spec for Waku Relay with RLN, alpha
|
||||
- [waku/2 swap](./specs/waku/v2/waku-swap-accounting) spec for Waku Swap Accounting, alpha
|
||||
- [waku/2 rpc api](./specs/waku/v2/waku-v2-rpc-api) - Waku RPC API for Waku v2 nodes, alpha
|
||||
|
||||
#### Version 0 and 1
|
||||
- [waku/0](./specs/waku/v1/waku-0) specs for `waku/0` version, now deprecated
|
||||
- [waku/1](./specs/waku/v1/waku-1) specs for `waku/1` version, current stable version
|
||||
- [envelope data format](./specs/waku/v1/envelope-data-format) [waku](./waku/waku) envelope data field specification.
|
||||
- [mailserver](./specs/waku/v1/mailserver) - Mailserver specification for archiving and delivering historical [waku](./specs/waku/waku) envelopes on demand.
|
||||
- [rpc api](./specs/waku/v1/waku-rpc-api) - Waku RPC API for Waku v1 nodes.
|
||||
|
||||
### Data sync
|
||||
|
||||
- [mvds](./specs/mvds) - Data Synchronization protocol for unreliable transports.
|
||||
- [remote log](./specs/remote-log) - Remote replication of local logs.
|
||||
- [mvds metadata](./specs/mvds-metadata) - Metadata field for [MVDS](./specs/mvds) messages.
|
||||
|
||||
## Style guide
|
||||
|
||||
Sequence diagrams are generated using [Mscgen](http://www.mcternan.me.uk/mscgen/) like this: `mscgen -T png -i input.msc -o output.png`. Both the source and generated image should be in source control. For ease of readability, the generated image is embedded inside the main spec document.
|
||||
|
||||
Alternatively, [mscgenjs](https://github.com/mscgenjs/mscgenjs-cli) can be used to generate sequence diagrams (mscgenjs produces better quality figures especially concerning lines' spaces and figures' margins). Once installed, the following command can be used to generate the sequence diagrams `mscgenjs -T png -i input.msc -o output.png`. More details on the installation and compilation are given in [mscgenjs repository](https://github.com/mscgenjs/mscgenjs-cli). You may try the online playground https://mscgen.js.org/ as well to get a sense of the output figures.
|
||||
|
||||
The lifecycle of the specs follows the [COSS Lifecycle](https://rfc.unprotocols.org/spec:2/COSS/)
|
||||
|
||||
## Meta
|
||||
|
||||
The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
|
||||
|
||||
## Linting
|
||||
|
||||
### Spellcheck
|
||||
|
||||
To run the spellchecker locally, you must install [pyspelling](https://facelessuser.github.io/pyspelling/).
|
||||
|
||||
It can then be run with the following command:
|
||||
|
||||
```console
|
||||
pyspelling -c .pyspelling.yml
|
||||
```
|
||||
|
||||
Words that should be ignored or are unrecognized must be added to the [wordlist](./wordlist.txt).
|
||||
|
||||
### Markdown Verification
|
||||
|
||||
We use [remark](https://remark.js.org/) to verify our markdown. You can easily run this tool simply by using our `npm` package:
|
||||
|
||||
```console
|
||||
npm install
|
||||
npm run lint
|
||||
```
|
||||
|
||||
### Textlint
|
||||
|
||||
We use [textlint](https://textlint.github.io/) for extra markdown verification. You can easily run this tool simply by using our `npm` package:
|
||||
|
||||
```console
|
||||
npm install
|
||||
npm run textlint
|
||||
```
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 24 KiB |
Before Width: | Height: | Size: 17 KiB After Width: | Height: | Size: 17 KiB |
Before Width: | Height: | Size: 122 KiB After Width: | Height: | Size: 122 KiB |
Before Width: | Height: | Size: 70 KiB After Width: | Height: | Size: 70 KiB |
Before Width: | Height: | Size: 50 KiB After Width: | Height: | Size: 50 KiB |
|
@ -0,0 +1,169 @@
|
|||
---
|
||||
domain: rfc.vac.dev
|
||||
shortname: 1/COSS
|
||||
name: Consensus-Oriented Specification System
|
||||
status: draft
|
||||
editor: Oskar Thoren <ot@oskarthoren.com>
|
||||
contributors:
|
||||
- Pieter Hintjens <ph@imatix.com>
|
||||
- André Rebentisch <andre@openstandards.de>
|
||||
- Alberto Barrionuevo <abarrio@opentia.es>
|
||||
- Chris Puttick <chris.puttick@thehumanjourney.net>
|
||||
- Yurii Rashkovskii <yrashk@gmail.com>
|
||||
---
|
||||
|
||||
This document describes a consensus-oriented specification system (COSS) for building interoperable technical specifications. COSS is based on a lightweight editorial process that seeks to engage the widest possible range of interested parties and move rapidly to consensus through working code.
|
||||
|
||||
This specification is based on [Unprotocols 2/COSS](https://github.com/unprotocols/rfc/blob/master/2/README.md), used by the [ZeromMQ](https://rfc.zeromq.org/) project. It is equivalent except for a few areas:
|
||||
|
||||
- Recommending the use of a permissive licenses, such as CC0 (with the exception of this document)
|
||||
- Misc metadata, editor, and format/link updates
|
||||
|
||||
## License
|
||||
|
||||
Copyright (c) 2008-20 the Editor and Contributors.
|
||||
|
||||
This Specification is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
|
||||
|
||||
This Specification is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License along with this program; if not, see http://www.gnu.org/licenses.
|
||||
|
||||
## Change Process
|
||||
|
||||
This document is governed by the [1/COSS](../1/README.md) (COSS).
|
||||
|
||||
## Language
|
||||
|
||||
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119)
|
||||
|
||||
## Goals
|
||||
|
||||
The primary goal of COSS is to facilitate the process of writing, proving, and improving new technical specifications. A "technical specification" defines a protocol, a process, an API, a use of language, a methodology, or any other aspect of a technical environment that can usefully be documented for the purposes of technical or social interoperability.
|
||||
|
||||
COSS is intended to above all be economical and rapid, so that it is useful to small teams with little time to spend on more formal processes.
|
||||
|
||||
Principles:
|
||||
|
||||
* We aim for rough consensus and running code.
|
||||
* Specifications are small pieces, made by small teams.
|
||||
* Specifications should have a clearly responsible editor.
|
||||
* The process should be visible, objective, and accessible to anyone.
|
||||
* The process should clearly separate experiments from solutions.
|
||||
* The process should allow deprecation of old specifications.
|
||||
|
||||
Specifications should take minutes to explain, hours to design, days to write, weeks to prove, months to become mature, and years to replace.
|
||||
|
||||
Specifications have no special status except that accorded by the community.
|
||||
|
||||
## Architecture
|
||||
|
||||
COSS is designed around fast, easy to use communications tools. Primarily, COSS uses a wiki model for editing and publishing specifications texts.
|
||||
|
||||
* The *domain* is the conservancy for a set of specifications in a certain area.
|
||||
* Each domain is implemented as an Internet domain, hosting a wiki and optionally other communications tools.
|
||||
* Each specification is a set of wiki pages, together with comments, attached files, and other resources.
|
||||
* Important specifications may also exist as subdomains, i.e. child wikis.
|
||||
|
||||
Individuals can become members of the domain by completing the necessary legal clearance. The copyright, patent, and trademark policies of the domain must be clarified in an Intellectual Property policy that applies to the domain.
|
||||
|
||||
Specifications exist as multiple pages, one page per version of the specification (see "Branching and Merging", below), which may be assigned URIs that include an incremental number. Thus, we refer to a specification by specifying its domain, number, and short name. New versions of the same specification will have new numbers. The syntax for a specification reference is:
|
||||
|
||||
<domain>/spec/<number>/<shortname>
|
||||
|
||||
For example, this specification is **rfc.vac.dev/spec/1/COSS**. The short form **1/COSS** may be used when referring to the specification from other specifications in the same domain.
|
||||
|
||||
Every specification (including branches) carries a different number. Lower numbers indicate more mature specifications, higher numbers indicate more experimental specifications.
|
||||
|
||||
## COSS Lifecycle
|
||||
|
||||
Every specification has an independent lifecycle that documents clearly its current status.
|
||||
|
||||
A specification has six possible states that reflect its maturity and contractual weight:
|
||||
|
||||
![Lifecycle diagram](../../../../assets/lifecycle.png)
|
||||
|
||||
### Raw Specifications
|
||||
All new specifications are **raw** specifications. Changes to raw specifications can be unilateral and arbitrary. Those seeking to implement a raw specification should ask for it to be made a draft specification. Raw specifications have no contractual weight.
|
||||
|
||||
### Draft Specifications
|
||||
|
||||
When raw specifications can be demonstrated, they become **draft** specifications. Changes to draft specifications should be done in consultation with users. Draft specifications are contracts between the editors and implementers.
|
||||
|
||||
### Stable Specifications
|
||||
When draft specifications are used by third parties, they become **stable** specifications. Changes to stable specifications should be restricted to cosmetic ones, errata and clarifications. Stable specifications are contracts between editors, implementers, and end-users.
|
||||
|
||||
### Deprecated Specifications
|
||||
|
||||
When stable specifications are replaced by newer draft specifications, they become **deprecated** specifications. Deprecated specifications should not be changed except to indicate their replacements, if any. Deprecated specifications are contracts between editors, implementers and end-users.
|
||||
|
||||
### Retired Specifications
|
||||
|
||||
When deprecated specifications are no longer used in products, they become **retired** specifications. Retired specifications are part of the historical record. They should not be changed except to indicate their replacements, if any. Retired specifications have no contractual weight.
|
||||
|
||||
### Deleted Specifications
|
||||
|
||||
Deleted specifications are those that have not reached maturity (stable) and were discarded. They should not be used and are only kept for their historical
|
||||
value. Only Raw and Draft specifications can be deleted.
|
||||
|
||||
## Editorial control
|
||||
|
||||
A specification MUST have a single responsible editor, the only person
|
||||
who SHALL change the status of the specification through the lifecycle stages.
|
||||
|
||||
A specification MAY also have additional contributors who contribute changes to it. It is RECOMMENDED to use a process similar to [C4 process](https://github.com/unprotocols/rfc/blob/master/1/README.md) to maximize the scale and diversity of contributions.
|
||||
|
||||
Unlike the original C4 process however, it is RECOMMENDED to use CC0 as a more permissive license alternative. We SHOULD NOT use GPL or GPL-like license. One exception is this specification, as this was the original license for this specification.
|
||||
|
||||
The editor is responsible for accurately maintaining the state of specifications and for handling all comments on the specification.
|
||||
|
||||
## Branching and Merging
|
||||
|
||||
Any member of the domain MAY branch a specification at any point. This is done by copying the existing text, and creating a new specification with the same name and content, but a new number. The ability to branch a specification is necessary in these circumstances:
|
||||
|
||||
* To change the responsible editor for a specification, with or without the cooperation of the current responsible editor.
|
||||
* To rejuvenate a specification that is stable but needs functional changes. This is the proper way to make a new version of a specification that is in stable or deprecated status.
|
||||
* To resolve disputes between different technical opinions.
|
||||
|
||||
The responsible editor of a branched specification is the person who makes the branch.
|
||||
|
||||
Branches, including added contributions, are derived works and thus licensed under the same terms as the original specification. This means that contributors are guaranteed the right to merge changes made in branches back into their original specifications.
|
||||
|
||||
Technically speaking, a branch is a *different* specification, even if it carries the same name. Branches have no special status except that accorded by the community.
|
||||
|
||||
## Conflict resolution
|
||||
|
||||
COSS resolves natural conflicts between teams and vendors by allowing anyone to define a new specification. There is no editorial control process except that practised by the editor of a new specification. The administrators of a domain (moderators) may choose to interfere in editorial conflicts, and may suspend or ban individuals for behaviour they consider inappropriate.
|
||||
|
||||
## Conventions
|
||||
|
||||
Where possible editors and contributors are encouraged to:
|
||||
|
||||
* Refer to and build on existing work when possible, especially IETF specifications.
|
||||
* Contribute to existing specifications rather than reinvent their own.
|
||||
* Use collaborative branching and merging as a tool for experimentation.
|
||||
|
||||
## Appendix A. Color Coding
|
||||
|
||||
It is RECOMMENDED to use color coding to indicate specification's status. Color coded specifications SHOULD use the following color scheme:
|
||||
|
||||
* ![raw](https://raw.githubusercontent.com/unprotocols/rfc/master/2/raw.svg)
|
||||
* ![draft](https://raw.githubusercontent.com/unprotocols/rfc/master/2/draft.svg)
|
||||
* ![stable](https://raw.githubusercontent.com/unprotocols/rfc/master/2/stable.svg)
|
||||
* ![deprecated](https://raw.githubusercontent.com/unprotocols/rfc/master/2/deprecated.svg)
|
||||
* ![retired](https://raw.githubusercontent.com/unprotocols/rfc/master/2/retired.svg)
|
||||
* ![deleted](https://raw.githubusercontent.com/unprotocols/rfc/master/2/deleted.svg)
|
||||
|
||||
## Appendix B. Metainformation
|
||||
|
||||
It is RECOMMENDED that specification metadata is specified as a YAML header (where possible) or a separate YAML file. This will enable programmatic access
|
||||
to specification metadata.
|
||||
|
||||
| Key | Value | Type | Example |
|
||||
|------------------|----------------------|--------|---------------------|
|
||||
| **domain** | specification domain | string | rfc.vac.dev |
|
||||
| **shortname** | short name | string | 1/COSS |
|
||||
| **name** | full name | string | Consensus-Oriented Specification System |
|
||||
| **status** | status | string | draft |
|
||||
| **editor** | editor name/email | string | Oskar Thoren <ot@oskarthoren.com> |
|
||||
| **contributors** | contributors | list | Pieter Hintjens <ph@imatix.com>, André Rebentisch <andre@openstandards.de>, Alberto Barrionuevo <abarrio@opentia.es>, Chris Puttick <chris.puttick@thehumanjourney.net>, Yurii Rashkovskii <yrashk@gmail.com> |
|
|
@ -0,0 +1,10 @@
|
|||
baseURL = "http://rfc.vac.dev"
|
||||
languageCode = "en-us"
|
||||
title = "Vac RFC"
|
||||
theme = "book"
|
||||
|
||||
[params]
|
||||
BookMenuBundle = "/menu"
|
||||
|
||||
[permalinks]
|
||||
docs = "/spec/:slug"
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
title: Introduction
|
||||
type: docs
|
||||
---
|
||||
|
||||
# Vac RFCs
|
||||
|
||||
{{< columns >}}
|
||||
## Specifications
|
||||
|
||||
Here you can find specifications for [Vac.dev](http://vac.dev/), a modular,
|
||||
peer-to-peer messaging stack for private, secure, censorship resistant
|
||||
communication.
|
||||
|
||||
<--->
|
||||
|
||||
## Contribute
|
||||
|
||||
Contributations are most welcome! First read [1/COSS](spec/1) then contribute at
|
||||
[Github](https://github.com/vacp2p/rfc.vac.dev).
|
||||
|
||||
{{< /columns >}}
|
||||
|
||||
|
||||
<!--
|
||||
|
||||
## Tempora nisi nunc
|
||||
|
||||
Lorem **markdownum** emicat gestu. Cannis sol pressit ducta. **Est** Idaei,
|
||||
tremens ausim se tutaeque, illi ulnis hausit, sed, lumina cutem. Quae avis
|
||||
sequens!
|
||||
|
||||
var panel = ram_design;
|
||||
if (backup + system) {
|
||||
file.readPoint = network_native;
|
||||
sidebar_engine_device(cell_tftp_raster,
|
||||
dual_login_paper.adf_vci.application_reader_design(
|
||||
graphicsNvramCdma, lpi_footer_snmp, integer_model));
|
||||
}
|
||||
|
||||
## Locis suis novi cum suoque decidit eadem
|
||||
|
||||
Idmoniae ripis, at aves, ali missa adest, ut _et autem_, et ab?
|
||||
|
||||
-->
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
domain: rfc.vac.dev
|
||||
shortname: 1/COSS
|
||||
slug: 1
|
||||
title: 1/COSS
|
||||
name: Consensus-Oriented Specification System
|
||||
status: draft
|
||||
editor: Oskar Thoren <ot@oskarthoren.com>
|
||||
|
@ -31,7 +31,7 @@ You should have received a copy of the GNU General Public License along with thi
|
|||
|
||||
## Change Process
|
||||
|
||||
This document is governed by the [1/COSS](../1/README.md) (COSS).
|
||||
This document is governed by the [1/COSS](spec/1) (COSS).
|
||||
|
||||
## Language
|
||||
|
||||
|
@ -81,7 +81,7 @@ Every specification has an independent lifecycle that documents clearly its curr
|
|||
|
||||
A specification has six possible states that reflect its maturity and contractual weight:
|
||||
|
||||
![Lifecycle diagram](../../../../assets/lifecycle.png)
|
||||
![Lifecycle diagram](../../../../rfcs/1/lifecycle.png)
|
||||
|
||||
### Raw Specifications
|
||||
All new specifications are **raw** specifications. Changes to raw specifications can be unilateral and arbitrary. Those seeking to implement a raw specification should ask for it to be made a draft specification. Raw specifications have no contractual weight.
|
||||
|
|
|
@ -0,0 +1,237 @@
|
|||
---
|
||||
slug: 10
|
||||
title: 10/WAKU2
|
||||
name: Waku v2
|
||||
status: draft
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
Waku is a privacy-preserving peer-to-peer messaging protocol for resource
|
||||
restricted devices. It implements PubSub over libp2p and adds capabilities on
|
||||
top of it. These capabilities are: (i) retrieving historical messages for
|
||||
mostly-offline devices (ii) adaptive nodes, allowing for heterogeneous nodes to
|
||||
contribute, and (iii) bandwidth preservation for light nodes. This makes it
|
||||
ideal for running a p2p protocol on mobile.
|
||||
|
||||
Historically, it has its roots in [Waku v1](/spec/6), which
|
||||
stems from [Whisper](https://eips.ethereum.org/EIPS/eip-627), originally part of
|
||||
the Ethereum stack. However, Waku v2 acts more as a thin wrapper for PubSub and
|
||||
has a different API. It is implemented in an iterative manner where initial
|
||||
focus is on porting essential functionality to libp2p. See [rough road
|
||||
map](https://vac.dev/waku-v2-plan).
|
||||
|
||||
# Motivation and goals
|
||||
|
||||
1. **Generalized messaging.** Many applications requires some form of messaging
|
||||
protocol to communicate between different subsystems or different nodes. This
|
||||
messaging can be human-to-human or machine-to-machine or a mix.
|
||||
|
||||
2. **Peer-to-peer.** These applications sometimes have requirements that make
|
||||
them suitable for peer-to-peer solutions.
|
||||
|
||||
3. **Resource restricted**.These applications often run in constrained
|
||||
environments, where resources or the environment is restricted in some
|
||||
fashion. E.g.:
|
||||
|
||||
- limited bandwidth, CPU, memory, disk, battery, etc
|
||||
- not being publicly connectable
|
||||
- only being intermittently connected; mostly-offline
|
||||
|
||||
4. **Privacy.** These applications have a desire for some privacy guarantees,
|
||||
such as pseudonymity, metadata protection in transit, etc.
|
||||
|
||||
Waku provides a solution that satisfies these goals in a reasonable matter.
|
||||
|
||||
# Network interaction domains
|
||||
|
||||
While Waku is best though of as a single cohesive thing, there are three network
|
||||
interaction domains: (a) gossip domain (b) discovery domain (c) req/resp domain.
|
||||
|
||||
## Protocols and identifiers
|
||||
|
||||
The current [protocol identifiers](https://docs.libp2p.io/concepts/protocols/) are:
|
||||
|
||||
1. `/vac/waku/relay/2.0.0-beta2`
|
||||
2. `/vac/waku/store/2.0.0-beta1`
|
||||
3. `/vac/waku/filter/2.0.0-beta1`
|
||||
|
||||
These protocols and their semantics are elaborated on in their own specs.
|
||||
|
||||
For the actual content being passed around, see the [7/WAKU-DATA](/spec/7) spec.
|
||||
|
||||
The WakuMessage spec being used is: `2.0.0-beta1`.
|
||||
|
||||
## Gossip domain
|
||||
|
||||
**Protocol identifier***: `/vac/waku/relay/2.0.0-beta1`
|
||||
|
||||
See [11/WAKU-RELAY](/spec/11) spec for more details.
|
||||
|
||||
### Default pubsub topic
|
||||
|
||||
The default PubSub topic being used for Waku is currently: `/waku/2/default-waku/proto`
|
||||
|
||||
This indicates that it relates to Waku, is version 2, is the default topic, and
|
||||
that the encoding of data field is protobuf.
|
||||
|
||||
The default PubSub topic SHOULD be used for all protocols. This ensures a
|
||||
connected network, as well some degree of metadata protection. It MAY be
|
||||
different if or when:
|
||||
<!-- TODO discuss anonymity related to one pubsub topic -->
|
||||
- Different applications have different message volume
|
||||
- Topic sharding is introduced
|
||||
- Encoding is changed
|
||||
- Version is changed
|
||||
|
||||
## Discovery domain
|
||||
|
||||
Discovery domain is not yet implemented. Currently static nodes should be used.
|
||||
|
||||
<!-- TODO: To document how we use Discovery v5, etc. See https://github.com/vacp2p/specs/issues/167 -->
|
||||
|
||||
## Request/reply domain
|
||||
|
||||
This consists of two main protocols. They are used in order to get Waku to run
|
||||
in resource restricted environments, such as low bandwidth or being mostly
|
||||
offline.
|
||||
|
||||
### Historical message support
|
||||
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta1`
|
||||
|
||||
See [13/WAKU-STORE](/spec/13) spec for more details.
|
||||
|
||||
### Content filtering
|
||||
|
||||
**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
|
||||
|
||||
See [14/WAKU-FILTER](/spec/14) spec for more details.
|
||||
|
||||
|
||||
# Upgradability and Compatibility
|
||||
|
||||
## Compatibility with Waku v1
|
||||
|
||||
Waku v2 and Waku v1 are different protocols all together. They use a different
|
||||
transport protocol underneath; Waku v1 is devp2p RLPx based while Waku v2 uses
|
||||
libp2p. The protocols themselves also differ as does their data format.
|
||||
Compatibility can be achieved only by using a bridge that not only talks both
|
||||
devp2p RLPx and libp2p, but that also transfers (partially) the content of a
|
||||
packet from one version to the other.
|
||||
|
||||
See [bridge spec](waku-bridge.md) for details on a bridge mode.
|
||||
|
||||
|
||||
# Security
|
||||
Each protocol layer of Waku v2 provides a distinct service and is associated with a separate set of security features and concerns. Therefore, the overall security of Waku v2 depends on how the different layers are utilized. In this section, we overview the security properties of Waku v2 protocols against a static adversarial model which is described below. Note that a more detailed security analysis of each Waku protocol is supplied in its respective specification as well.
|
||||
|
||||
## Primary Adversarial Model
|
||||
In the primary adversarial model, we consider adversary as a passive entity that attempts to collect information from others to conduct an attack but it does so without violating protocol definitions and instructions.
|
||||
|
||||
The following are **not** considered as part of the adversarial model:
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
|
||||
|
||||
## Security Features
|
||||
|
||||
### Pseudonymity
|
||||
Waku v2 by default guarantees pseudonymity for all of the protocol layers since parties do not have to disclose their true identity and instead they utilize libp2p `PeerID` as their identifiers. While pseudonymity is an appealing security feature, it does not guarantee full anonymity since the actions taken under the same pseudonym i.e., `PeerID` can be linked together and potentially result in the re-identification of the true actor.
|
||||
|
||||
### Anonymity / Unlinkability
|
||||
At a high level, anonymity is the inability of an adversary in linking an actor to its data/performed action (the actor and action are context-dependent). To be precise about linkability, we use the term Personally Identifiable Information (PII) to refer to any piece of data that could potentially be used to uniquely identify a party. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII. Notice that users' actions can be traced through their PIIs (e.g., signatures) and hence result in their re-identification risk. As such, we seek anonymity by avoiding linkability between actions and the actors / actors' PII. Concerning anonymity, Waku v2 provides the following features:
|
||||
|
||||
**Publisher-Message Unlinkability**: This feature signifies the unlinkability of a publisher to its published messages in the `WakuRelay` protocol. The [Publisher-Message Unlinkability](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-relay.md#security-analysis) is enforced through the `StrictNoSign` policy due to which the data fields of pubsub messages that count as PII for the publisher must be left unspecified.
|
||||
|
||||
**Subscriber-Topic Unlinkability**: This feature stands for the unlinkability of the subscriber to its subscribed topics in the `WakuRelay` protocol. The [Subscriber-Topic Unlinkability](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-relay.md#security-analysis) is achieved through the utilization of a single pubsub topic. As such, subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID. This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of subscribers). Note that there is no hard limit on the number of the pubsub topics, however, the use of one topic is recommended for the sake of anonymity.
|
||||
|
||||
### Spam protection
|
||||
This property indicates that no adversary can flood the system (i.e., publishing a large number of messages in a short amount of time), either accidentally or deliberately, with any kind of message i.e. even if the message content is valid or useful. Spam protection is partly provided in `WakuRelay` through [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures) of GossipSub v1.1. At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score.
|
||||
|
||||
### Data confidentiality, Integrity, Authenticity
|
||||
Confidentiality can be addressed through data encryption whereas integrity and authenticity are achievable through digital signatures. These features are enabled in `WakuMessage` v1 through payload encryption as well as encrypted signatures (see [WakuMessage specs](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-message.md#version-1-not-yet-implemented-in-waku-v2) for further details).
|
||||
|
||||
## Security Considerations
|
||||
|
||||
**Lack of anonymity/unlinkability in the protocols involving direct connections including `WakuStore` and `WakuFilter` protocols**: The anonymity/unlinkability is not guaranteed in the protocols like `WakuStore` and `WakuFilter` where peers need to have direct connections to benefit from the designated service. This is because during the direct connections peers utilize `PeerID` to identify each other, therefore the service obtained in the protocol is linkable to the beneficiary's `PeerID` (which counts as PII). In terms of `WakuStore`, the queried node would be able to link the querying node's `PeerID` to its queried topics. Likewise, in the `WakuFilter`, a full node can link the light node's `PeerID`s to its content filter. <!-- TODO: to inspect the nimlibp2p codebase and figure out the exact use of PeerIDs in direct communication, it might be the case that the requester does not have to disclose its PeerID-->
|
||||
|
||||
<!--TODO: might be good to add a figure visualizing the Waku protocol stack and the security features of each layer-->
|
||||
|
||||
## Future work
|
||||
|
||||
We are actively working on the following features to be added to Waku v2.
|
||||
|
||||
**Economic Spam resistance**: We aim to enable an incentivized spam protection technique at the `WakuRelay` by using rate limiting nullifiers. More details on this can be found in [Waku RLN Relay](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-rln-relay.md). In this advanced method, peers are limited to a certain rate of messaging per epoch and an immediate financial penalty is enforced for spammers who break this rate.
|
||||
|
||||
**Prevention of Denial of Service (DoS)**: Denial of service signifies the case where an adversarial node exhausts another node's service capacity (e.g., by making a large number of requests) and makes it unavailable to the rest of the system. DoS attack is to be mitigated through the accounting model as described in [Waku Swap Accounting specs](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-swap-accounting.md). In a nutshell, peers have to pay for the service they obtain from each other. In addition to incentivizing the service provider, accounting also makes DoS attacks costly for malicious peers. The accounting model can be used in `WakuStore` and `WakuFilter` to protect against DoS attacks.
|
||||
|
||||
# Changelog
|
||||
|
||||
### Next version
|
||||
|
||||
- Add recommended default PubSub topic
|
||||
- Update relay and message spec version
|
||||
- Security analysis under the primary adversarial model is done
|
||||
|
||||
### 2.0.0-beta1
|
||||
|
||||
Initial draft version. Released [2020-09-17](https://github.com/vacp2p/specs/commit/a57dad2cc3d62f9128e21f68719704a0b358768b)
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
# References
|
||||
|
||||
1. [Protocol Identifiers](https://docs.libp2p.io/concepts/protocols/)
|
||||
|
||||
2. [PubSub interface for libp2p (r2,
|
||||
2019-02-01)](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
|
||||
|
||||
3. [GossipSub
|
||||
v1.0](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md)
|
||||
|
||||
4. [GossipSub
|
||||
v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
|
||||
|
||||
5. [Waku v1 spec](/spec/6)
|
||||
|
||||
6. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)
|
||||
|
||||
7. [Waku v2 plan](https://vac.dev/waku-v2-plan)
|
||||
|
||||
8. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
|
||||
|
||||
9. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)
|
||||
|
||||
<!--
|
||||
|
||||
# Underlying transports, etc
|
||||
|
||||
## Peer Discovery
|
||||
|
||||
WakuSub and PubSub don't provide an peer discovery mechanism. This has to be
|
||||
provided for by the environment.
|
||||
|
||||
### PubSub interface
|
||||
|
||||
Waku v2 is implementing the PubSub interface in Libp2p. See [PubSub interface
|
||||
for libp2p (r2, 2019-02-01)](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
|
||||
for
|
||||
more details.
|
||||
|
||||
### FloodSub
|
||||
|
||||
WakuSub is currently a subprotocol of FloodSub. Future versions of WakuSub will
|
||||
support [GossipSub v1.0](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md)
|
||||
and [GossipSub 1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md).
|
||||
|
||||
### Bridge mode
|
||||
|
||||
To maintain compatibility with Waku v1, a bridge mode can be achieved. See
|
||||
separate spec.
|
||||
|
||||
TODO Detail this in a separate spec
|
||||
-->
|
|
@ -0,0 +1,171 @@
|
|||
---
|
||||
slug: 11
|
||||
title: 11/WAKU2-RELAY
|
||||
name: Waku v2 Relay
|
||||
status: draft
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
`WakuRelay` is part of the gossip domain for Waku. It is a thin layer on top of GossipSub.
|
||||
|
||||
**Protocol identifier***: `/vac/waku/relay/2.0.0-beta2`
|
||||
|
||||
# Security Requirements
|
||||
|
||||
The `WakuRelay` protocol is designed to provide the following security properties under a static [Adversarial Model](#adversarial-model). Note that data confidentiality, integrity, and authenticity are currently considered out of the scope of the `WakuRelay` protocol and must be handled by the upper-level protocols such as [WakuMessage](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-message.md).
|
||||
|
||||
<!-- May add the definition of the unsupported feature:
|
||||
Confidentiality indicates that an adversary should not be able to learn the data carried by the `WakuRelay` protocol.
|
||||
Integrity indicates that the data transferred by the `WakuRelay` protocol can not be tampered with by an adversarial entity without being detected.
|
||||
Authenticity no adversary can forge data on behalf of a targeted publisher and make it accepted by other subscribers as if the origin is the target. -->
|
||||
|
||||
|
||||
- **Publisher-Message Unlinkability**: This property indicates that no adversarial entity can link a published `Message` to its publisher. This feature also implies the unlinkability of the publisher to its published topic ID as the `Message` embodies the topic IDs.
|
||||
|
||||
- **Subscriber-Topic Unlinkability**: This feature stands for the inability of any adversarial entity from linking a subscriber to its subscribed topic IDs.
|
||||
|
||||
|
||||
<!-- TODO: more requirements can be added, but that needs further and deeper investigation-->
|
||||
|
||||
|
||||
## Terminology
|
||||
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
# Adversarial Model
|
||||
- Any entity running the `WakuRelay` protocol is considered an adversary. This includes publishers, subscribers, and all the peers' direct connections. Furthermore, we consider the adversary as a passive entity that attempts to collect information from others to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious subscriber hides the messages it receives from other subscribers as it is against the description of the `WakuRelay` protocol. However, a malicious subscriber may learn which topics are subscribed to by which peers.
|
||||
- The following are **not** considered as part of the adversarial model:
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
|
||||
|
||||
|
||||
# Wire Specification
|
||||
|
||||
We are using protobuf RPC messages between peers. Here's what the protobuf messages look like, as defined in the PubSub interface. Please see [PubSub interface spec](https://github.com/libp2p/specs/blob/master/pubsub/README.md) for more details.
|
||||
|
||||
In this section, we specify two things how WakuSub is using these messages.
|
||||
|
||||
## Protobuf
|
||||
|
||||
```protobuf
|
||||
message RPC {
|
||||
repeated SubOpts subscriptions = 1;
|
||||
repeated Message publish = 2;
|
||||
|
||||
message SubOpts {
|
||||
optional bool subscribe = 1;
|
||||
optional string topicid = 2;
|
||||
}
|
||||
|
||||
message Message {
|
||||
optional string from = 1;
|
||||
optional bytes data = 2;
|
||||
optional bytes seqno = 3;
|
||||
repeated string topicIDs = 4;
|
||||
optional bytes signature = 5;
|
||||
optional bytes key = 6;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
WakuRelay does not currently use the `ControlMessage` defined in GossipSub.
|
||||
However, later versions will add likely add this capability.
|
||||
|
||||
`TopicDescriptor` as defined in the PubSub interface spec is not currently used.
|
||||
|
||||
## RPC
|
||||
|
||||
These are messages sent to directly connected peers. They SHOULD NOT be
|
||||
gossiped. See section below on how the fields work.
|
||||
|
||||
## Message
|
||||
|
||||
The `from` field MAY indicate which peer is publishing the message.
|
||||
|
||||
The `data` field MUST be filled out with a `WakuMessage`. See the [Waku Message](waku-message.md) spec for more details.
|
||||
|
||||
The `seqno` field MAY be used to provide a linearly increasing number. See PubSub spec for more details.
|
||||
|
||||
The `topicIDs` field MUST contain the topics that a message is being published on.
|
||||
|
||||
The `signature` field MAY contain the signature of the message, thus providing authentication of the message. See PubSub spec for more details.
|
||||
|
||||
The `key` field MAY be present and relates to signing. See PubSub spec for more details.
|
||||
|
||||
|
||||
## SubOpts
|
||||
|
||||
To do topic subscription management, we MAY send updates using `SubOpts` to our peers. If we do so, then:
|
||||
|
||||
The `subscribe` field MUST contain a boolean, where 1 means subscribe and 0 means unsubscribe to a topic.
|
||||
|
||||
The `topicid` field MUST contain the topic.
|
||||
|
||||
## Signature Policy
|
||||
|
||||
The `StrictNoSign` option MUST be used.
|
||||
|
||||
# Security Analysis
|
||||
<!-- TODO: realized that the prime security objective of the `WakuRelay` protocol is to provide peers unlinkability as such this feature is prioritized over other features e.g., unlinkability is preferred over authenticity and integrity. It might be good to motivate unlinkability and its impact on the relay protocol or other protocols invoking relay protocol.-->
|
||||
|
||||
- **Publisher-Message Unlinkability**: To address publisher-message unlinkability, one should remove any PII from the published message. As such, `WakuRelay` protocol follows the `StrictNoSign` policy as described in [libp2p PubSub specs](https://github.com/libp2p/specs/tree/master/pubsub#message-signing). As the result of the `StrictNoSign` policy, `Message`s should be built without the `from`, `signature` and `key` fields since each of these three fields individually counts as PII for the author of the message (one can link the creation of the message with libp2p peerId and thus indirectly with the IP address of the publisher).
|
||||
Note that removing identifiable information from messages cannot lead to perfect unlinkability. The direct connections of a publisher might be able to figure out which `Message`s belong to that publisher by analyzing its traffic. The possibility of such inference may get higher when the `data` field is also not encrypted by the upper-level protocols. <!-- TODO: more investigation on traffic analysis attacks and their success probability-->
|
||||
|
||||
- **Subscriber-Topic Unlinkability:** To preserve subscriber-topic unlinkability, it is recommended by [Waku v2 specs](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-v2.md#default-pubsub-topic) to use a single PubSub topic in the `WakuRelay` protocol. This allows an immediate subscriber-topic unlinkability where subscribers are not re-identifiable from their subscribed topic IDs as the entire network is linked to the same topic ID. This level of unlinkability / anonymity is known as [k-anonymity](https://www.privitar.com/blog/k-anonymity-an-introduction/) where k is proportional to the system size (number of subscribers).
|
||||
However, note that the `WakuRelay` supports the use of more than one topic. In that case, the unlinkability should be addressed by the upper-level protocols through e.g., [Partitioned topics technique](https://specs.status.im/spec/10#partitioned-topic) which enables K-anonymity for the peers' subscribed topic ids.
|
||||
|
||||
# Future work
|
||||
|
||||
- **Economic Spam resistant**: In the spam-protected `WakuRelay` protocol, no adversary can flood the system with spam messages (i.e., publishing a large number of messages in a short amount of time). Spam protection is partly provided by GossipSub v1.1 through [scoring mechanism](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#spam-protection-measures). At a high level, peers utilize a scoring function to locally score the behavior of their connections and remove peers with a low score. The `WakuRelay` protocol aims at enabling an advanced spam protection mechanism with economic disincentives by utilizing Rate Limiting Nullifiers. In a nutshell, peers must conform to a certain message publishing rate per a system-defined epoch, otherwise, they get financially penalized for exceeding the rate. More details on this new technique can be found in [Waku RLN Relay](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-rln-relay.md).
|
||||
<!-- TODO havn't checked if all the measures in libp2p GossipSub v1.1 are taken in the nim-libp2p as well, may need to audit the code -->
|
||||
|
||||
|
||||
- Providing **Unlinkability**, **Integrity** and **Authenticity** simultaneously: Integrity and authenticity are typically addressed through digital signatures and Message Authentication Code (MAC) schemes, however, the usage of digital signatures (where each signature is bound to a particular peer) contradicts with the unlinkability requirement (messages signed under a certain signature key are verifiable by a verification key that is bound to a particular publisher). As such, integrity and authenticity are missing features in the `WakuRelay` protocol in the interest of unlinkability. In future work, advanced signature schemes like group signatures can be utilized to enable authenticity, integrity, and unlinkability simultaneously. In a group signature scheme, a member of a group can anonymously sign a message on behalf of the group as such the true signer is indistinguishable from other group members. <!-- TODO: shall I add a reference for group signatures?-->
|
||||
|
||||
|
||||
# Changelog
|
||||
|
||||
### Next
|
||||
- Added initial threat model and security analysis
|
||||
### 2.0.0-beta2
|
||||
|
||||
Next version. Changes:
|
||||
|
||||
- Moved WakuMessage to separate spec and made it mandatory
|
||||
- StrictNoSign
|
||||
|
||||
|
||||
### 2.0.0-beta1
|
||||
|
||||
Initial draft version. Released [2020-09-17](https://github.com/vacp2p/specs/commit/a57dad2cc3d62f9128e21f68719704a0b358768b)
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
# References
|
||||
|
||||
1. [PubSub interface for libp2p (r2,
|
||||
2019-02-01)](https://github.com/libp2p/specs/blob/master/pubsub/README.md)
|
||||
|
||||
2. [GossipSub
|
||||
v1.0](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md)
|
||||
|
||||
3. [GossipSub
|
||||
v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md)
|
||||
|
||||
4. [Waku v1 spec](https://specs.vac.dev/waku/waku.html)
|
||||
|
||||
5. [Whisper spec (EIP627)](https://eips.ethereum.org/EIPS/eip-627)
|
||||
|
||||
|
||||
<!--
|
||||
TODO: Don't quite understand this scenario [key field], to clarify. Wouldn't it always be in `from`?
|
||||
> The key field contains the signing key when it cannot be inlined in the source peer ID. When present, it must match the peer ID. -->
|
||||
|
||||
Re topicid:
|
||||
NOTE: This doesn't appear to be documented in PubSub spec, upstream?
|
||||
-->
|
|
@ -0,0 +1,167 @@
|
|||
---
|
||||
slug: 12
|
||||
title: 12/WAKU2-FILTER
|
||||
name: Waku v2 Filter
|
||||
status: draft
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
`WakuFilter` is a protocol that enables subscribing to messages that a peer receives. This is a more lightweight version of `WakuRelay` specifically designed for bandwidth restricted devices. This is due to the fact that light nodes subscribe to full-nodes and only receive the messages they desire.
|
||||
|
||||
# Content filtering
|
||||
|
||||
**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
|
||||
|
||||
Content filtering is a way to do [message-based
|
||||
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
|
||||
Currently the only content filter being applied is on `contentTopic`. This
|
||||
corresponds to topics in Waku v1.
|
||||
|
||||
## Rationale
|
||||
|
||||
Unlike the `store` protocol for historical messages, this protocol allows for
|
||||
native lower latency scenarios such as instant messaging. It is thus
|
||||
complementary to it.
|
||||
|
||||
Strictly speaking, it is not just doing basic request response, but performs
|
||||
sender push based on receiver intent. While this can be seen as a form of light
|
||||
pub/sub, it is only used between two nodes in a direct fashion. Unlike the
|
||||
Gossip domain, this is meant for light nodes which put a premium on bandwidth.
|
||||
No gossiping takes place.
|
||||
|
||||
It is worth noting that a light node could get by with only using the `store`
|
||||
protocol to query for a recent time window, provided it is acceptable to do
|
||||
frequent polling.
|
||||
|
||||
|
||||
# Design Requirements
|
||||
|
||||
The effectiveness and reliability of the content filtering service enabled by `WakuFilter` protocol rely on the *high availability* of the full nodes as the service providers. To this end, full nodes must feature *high uptime* (to persistently listen and capture the network messages) as well as *high Bandwidth* (to provide timely message delivery to the light nodes).
|
||||
|
||||
# Security Consideration
|
||||
|
||||
Note that while using `WakuFilter` allows light nodes to save bandwidth, it comes with a privacy cost in the sense that they need to disclose their liking topics to the full nodes to retrieve the relevant messages. Currently, anonymous subscription is not supported by the `WakuFilter`, however, potential solutions in this regard are sketched below in [Future Work](#future-work) section.
|
||||
|
||||
## Terminology
|
||||
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
# Adversarial Model
|
||||
Any node running the `WakuFilter` protocol i.e., both the subscriber node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other nodes to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node intentionally hides the messages matching to one's subscribed content filter as it is against the description of the `WakuFilter` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
- An adversary with a global view of all the nodes and their connections.
|
||||
- An adversary that can eavesdrop on communication links between arbitrary pairs of nodes (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
|
||||
|
||||
## Protobuf
|
||||
|
||||
```protobuf
|
||||
message FilterRequest {
|
||||
bool subscribe = 1;
|
||||
string topic = 2;
|
||||
repeated ContentFilter contentFilters = 3;
|
||||
|
||||
message ContentFilter {
|
||||
repeated uint32 contentTopics = 1;
|
||||
}
|
||||
}
|
||||
|
||||
message MessagePush {
|
||||
repeated WakuMessage messages = 1;
|
||||
}
|
||||
|
||||
message FilterRPC {
|
||||
string requestId = 1;
|
||||
FilterRequest request = 2;
|
||||
MessagePush push = 3;
|
||||
}
|
||||
```
|
||||
|
||||
#### FilterRPC
|
||||
|
||||
A node MUST send all Filter messages (`FilterRequest`, `MessagePush`) wrapped inside a
|
||||
`FilterRPC` this allows the node handler to determine how to handle a message as the Waku
|
||||
Filter protocol is not a request response based protocol but instead a push based system.
|
||||
|
||||
The `requestId` MUST be a uniquely generated string. When a `MessagePush` is sent
|
||||
the `requestId` MUST match the `requestId` of the subscribing `FilterRequest` whose filters
|
||||
matched the message causing it to be pushed.
|
||||
|
||||
#### FilterRequest
|
||||
|
||||
A `FilterRequest` contains an optional topic, zero or more content filters and
|
||||
a boolean signifying whether to subscribe or unsubscribe to the given filters.
|
||||
True signifies 'subscribe' and false signifies 'unsubscribe'.
|
||||
|
||||
A node that sends the RPC with a filter request and `subscribe` set to 'true'
|
||||
requests that the filter node SHOULD notify the light requesting node of messages
|
||||
matching this filter.
|
||||
|
||||
A node that sends the RPC with a filter request and `subscribe` set to 'false'
|
||||
requests that the filter node SHOULD stop notifying the light requesting node
|
||||
of messages matching this filter if it is currently doing so.
|
||||
|
||||
The filter matches when content filter and, optionally, a topic is matched.
|
||||
Content filter is matched when a `WakuMessage` `contentTopic` field is the same.
|
||||
|
||||
A filter node SHOULD honor this request, though it MAY choose not to do so. If
|
||||
it chooses not to do so it MAY tell the light why. The mechanism for doing this
|
||||
is currently not specified. For notifying the light node a filter node sends a
|
||||
MessagePush message.
|
||||
|
||||
Since such a filter node is doing extra work for a light node, it MAY also
|
||||
account for usage and be selective in how much service it provides. This
|
||||
mechanism is currently planned but underspecified.
|
||||
|
||||
#### MessagePush
|
||||
|
||||
A filter node that has received a filter request SHOULD push all messages that
|
||||
match this filter to a light node. These [`WakuMessage`'s](./waku-message.md) are likely to come from the
|
||||
`relay` protocol and be kept at the Node, but there MAY be other sources or
|
||||
protocols where this comes from. This is up to the consumer of the protocol.
|
||||
|
||||
A filter node MUST NOT send a push message for messages that have not been
|
||||
requested via a FilterRequest.
|
||||
|
||||
If a specific light node isn't connected to a filter node for some specific
|
||||
period of time (e.g. a TTL), then the filter node MAY choose to not push these
|
||||
messages to the node. This period is up to the consumer of the protocol and node
|
||||
implementation, though a reasonable default is one minute.
|
||||
|
||||
---
|
||||
# Future Work
|
||||
<!-- Alternative title: Filter-subscriber unlinkability -->
|
||||
**Anonymous filter subscription**: This feature guarantees that nodes can anonymously subscribe for a message filter (i.e., without revealing their exact content filter). As such, no adversary in the `WakuFilter` protocol would be able to link nodes to their subscribed content filers. The current version of the `WakuFilter` protocol does not provide anonymity as the subscribing node has a direct connection to the full node and explicitly submits its content filter to be notified about the matching messages. However, one can consider preserving anonymity through one of the following ways:
|
||||
- By hiding the source of the subscription i.e., anonymous communication. That is the subscribing node shall hide all its PII in its filter request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor<!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the WakuFilter protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a subscriber and its content filter despite hiding the IP address-->.
|
||||
Note that the current structure of filter requests i.e., `FilterRPC` does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve anonymity.
|
||||
- By deploying secure 2-party computations in which the subscribing node obtains the messages matching a content filter whereas the full node learns nothing about the content filter as well as the messages pushed to the subscribing node. Examples of such 2PC protocols are [Oblivious Transfers](https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-5906-5_9#:~:text=Oblivious%20transfer%20(OT)%20is%20a,information%20the%20receiver%20actually%20obtains.) and one-way Private Set Intersections (PSI).
|
||||
|
||||
# Changelog
|
||||
|
||||
### Next
|
||||
|
||||
- Added initial threat model and security analysis.
|
||||
|
||||
### 2.0.0-beta2
|
||||
|
||||
Initial draft version. Released [2020-10-28](https://github.com/vacp2p/specs/commit/5ceeb88cee7b918bb58f38e7c4de5d581ff31e68)
|
||||
- Fix: Ensure contentFilter and contentTopic are repeated fields, per implementation
|
||||
- Change: Add ability to unsubscribe from filters. Make `subscribe` an explicit boolean indication. Edit protobuf field order to be consistent with libp2p.
|
||||
|
||||
### 2.0.0-beta1
|
||||
|
||||
Initial draft version. Released [2020-10-05](https://github.com/vacp2p/specs/commit/31857c7434fa17efc00e3cd648d90448797d107b)
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
# References
|
||||
|
||||
1. [Message Filtering (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering)
|
||||
|
||||
2. [Libp2p PubSub spec - topic validation](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation)
|
|
@ -0,0 +1,121 @@
|
|||
---
|
||||
slug: 13
|
||||
title: 13/WAKU2-STORE
|
||||
name: Waku v2 Store
|
||||
status: draft
|
||||
editor: Sanaz Taheri <sanaz@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
This specification explains the Waku `WakuStore` protocol which enables querying of messages received through relay protocol and stored by other nodes. It also supports pagination for more efficient querying of historical messages.
|
||||
|
||||
**Protocol identifier***: `/vac/waku/store/2.0.0-beta2`
|
||||
|
||||
# Design Requirements
|
||||
Nodes willing to provide storage service in `WakuStore` protocol are required to be *highly available* and in specific have a *high uptime* to consistently receive and store network messages. The high uptime requirement makes sure that no message is missed out hence a complete and intact view of the message history is delivered to the querying nodes. Nevertheless, in case that storage provider nodes cannot afford high availability, the querying nodes may retrieve the historical messages from multiple sources to achieve a full and intact view of the past.
|
||||
|
||||
# Security Consideration
|
||||
The main security consideration to be taken into account while using `WakuStore` is to notice that the querying nodes have to reveal their topics of interest to the queried nodes hence compromising their privacy.
|
||||
|
||||
## Terminology
|
||||
The term Personally identifiable information (PII) refers to any piece of data that can be used to uniquely identify a user. For example, the signature verification key, and the hash of one's static IP address are unique for each user and hence count as PII.
|
||||
|
||||
# Adversarial Model
|
||||
Any peer running the `WakuStore` protocol i.e., both the querying node and the queried node are considered as an adversary. Furthermore, we consider the adversary as a passive entity that attempts to collect information from other peers to conduct an attack but it does so without violating protocol definitions and instructions. For example, under the passive adversarial model, no malicious node hides or lies about the history of messages as it is against the description of the `WakuStore` protocol.
|
||||
|
||||
The following are not considered as part of the adversarial model:
|
||||
- An adversary with a global view of all the peers and their connections.
|
||||
- An adversary that can eavesdrop on communication links between arbitrary pairs of peers (unless the adversary is one end of the communication). In specific, the communication channels are assumed to be secure.
|
||||
|
||||
|
||||
# Wire Specification
|
||||
Peers communicate with each other using a request / response API. The messages sent are Protobuf RPC messages. The followings are the specifications of the Protobuf messages.
|
||||
|
||||
## Protobuf
|
||||
|
||||
```protobuf
|
||||
message Index {
|
||||
bytes digest = 1;
|
||||
float receivedTime = 2;
|
||||
}
|
||||
|
||||
message PagingInfo {
|
||||
int64 pageSize = 1;
|
||||
Index cursor = 2;
|
||||
enum Direction {
|
||||
FORWARD = 0;
|
||||
BACKWARD = 1;
|
||||
}
|
||||
Direction direction = 3;
|
||||
}
|
||||
|
||||
message HistoryQuery {
|
||||
repeated string topics = 2;
|
||||
optional PagingInfo pagingInfo = 3; // used for pagination
|
||||
}
|
||||
|
||||
message HistoryResponse {
|
||||
repeated WakuMessage messages = 2;
|
||||
optional PagingInfo pagingInfo = 3; // used for pagination
|
||||
}
|
||||
|
||||
message HistoryRPC {
|
||||
string request_id = 1;
|
||||
HistoryQuery query = 2;
|
||||
HistoryResponse response = 3;
|
||||
}
|
||||
```
|
||||
|
||||
### Index
|
||||
|
||||
To perform pagination, each `WakuMessage` stored at a node running the `WakuStore` protocol is associated with a unique `Index` that encapsulates the following parts.
|
||||
- `digest`: a sequence of bytes representing the hash of a `WakuMessage`.
|
||||
- `receivedTime`: the UNIX time at which the waku message is received by the node running the `WakuStore` protocol.
|
||||
|
||||
### PagingInfo
|
||||
|
||||
`PagingInfo` holds the information required for pagination. It consists of the following components.
|
||||
- `pageSize`: A positive integer indicating the number of queried `WakuMessage`s in a `HistoryQuery` (or retrieved `WakuMessage`s in a `HistoryResponse`).
|
||||
- `cursor`: holds the `Index` of a `WakuMessage`.
|
||||
- `direction`: indicates the direction of paging which can be either `FORWARD` or `BACKWARD`.
|
||||
|
||||
### HistoryQuery
|
||||
|
||||
RPC call to query historical messages.
|
||||
|
||||
- The `topics` field MUST indicate the list of topics to query.
|
||||
- `PagingInfo` holds the information required for pagination. Its `pageSize` field indicates the number of `WakuMessage`s to be included in the corresponding `HistoryResponse`. If the `pageSize` is zero then no pagination is required. If the `pageSize` exceeds a threshold then the threshold value shall be used instead. In the forward pagination request, the `messages` field of the `HistoryResponse` shall contain at maximum the `pageSize` amount of waku messages whose `Index` values are larger than the given `cursor` (and vise versa for the backward pagination). Note that the `cursor` of a `HistoryQuery` may be empty (e.g., for the initial query), as such, and depending on whether the `direction` is `BACKWARD` or `FORWARD` the last or the first `pageSize` waku messages shall be returned, respectively.
|
||||
The queried node MAY sort the `WakuMessage`s based on their `Index`, where the `receivedTime` constitutes the most significant part and the `digest` comes next, and then perform pagination on the sorted result. As such, the retrieved page contains an ordered list of `WakuMessage`s from the oldest message to the most recent one.
|
||||
|
||||
### HistoryResponse
|
||||
|
||||
RPC call to respond to a HistoryQuery call.
|
||||
- The `messages` field MUST contain the messages found, these are [`WakuMessage`] types as defined in the corresponding [specification](./waku-message.md).
|
||||
- `PagingInfo` holds the paging information based on which the querying node can resume its further history queries. The `pageSize` indicates the number of returned waku messages (i.e., the number of messages included in the `messages` field of `HistoryResponse`). The `direction` is the same direction as in the corresponding `HistoryQuery`. In the forward pagination, the `cursor` holds the `Index` of the last message in the `HistoryResponse` `messages` (and the first message in the backward paging). The requester shall embed the returned `cursor` inside its next `HistoryQuery` to retrieve the next page of the waku messages. The `cursor` obtained from one node SHOULD NOT be used in a request to another node because the result MAY be different.
|
||||
|
||||
# Future Work
|
||||
|
||||
- **Anonymous query**: This feature guarantees that nodes can anonymously query historical messages from other nodes (i.e., without disclosing the exact topics of waku messages they are interested in). As such, no adversary in the `WakuStore` protocol would be able to learn which peer is interested in which topics of waku message. The current version of the `WakuStore` protocol does not provide anonymity for historical queries as the querying node needs to directly connect to another node in the `WakuStore` protocol and explicitly disclose the topics of its interest to retrieve the corresponding messages. However, one can consider preserving anonymity through one of the following ways:
|
||||
- By hiding the source of the request i.e., anonymous communication. That is the querying node shall hide all its PII in its history request e.g., its IP address. This can happen by the utilization of a proxy server or by using Tor. Note that the current structure of historical requests does not embody any piece of PII, otherwise, such data fields must be treated carefully to achieve query anonymity. <!-- TODO: if nodes have to disclose their PeerIDs (e.g., for authentication purposes) when connecting to other nodes in the store protocol, then Tor does not preserve anonymity since it only helps in hiding the IP. So, the PeerId usage in switches must be investigated further. Depending on how PeerId is used, one may be able to link between a querying node and its queried topics despite hiding the IP address-->.
|
||||
- By deploying secure 2-party computations in which the querying node obtains the historical messages of a certain topic whereas the queried node learns nothing about the query. Examples of such 2PC protocols are secure one-way Private Set Intersections (PSI). <!-- TODO: add a reference for PSIs? --> <!-- TODO: more techniques to be included -->.
|
||||
<!-- TODO: Censorship resistant: this is about a node that hides the historical messages from other nodes. This attack is not included in the specs since it does not fit the passive adversarial model (the attacker needs to deviate from the store protocol).-->
|
||||
|
||||
# Changelog
|
||||
|
||||
### Next
|
||||
- Added the initial threat model and security analysis.
|
||||
|
||||
### 2.0.0-beta2
|
||||
Released [2020-11-05](https://github.com/vacp2p/specs/commit/edc90625ffb5ce84cc6eb6ec4ec1a99385fad125)
|
||||
- Added pagination support.
|
||||
|
||||
### 2.0.0-beta1
|
||||
Released [2020-10-06](https://github.com/vacp2p/specs/commit/75b4c39e7945eb71ad3f9a0a62b99cff5dac42cf)
|
||||
- Initial draft version.
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
slug: 14
|
||||
title: 14/WAKU2-MESSAGE
|
||||
name: Waku v2 Message
|
||||
status: draft
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
This specification provides a way to encapsulate messages sent over Waku with specific information security goals.
|
||||
|
||||
# Motivation
|
||||
|
||||
When using Waku to send messages over Waku there are multiple concerns:
|
||||
- We may have a separate encryption layer as part of our application
|
||||
- We may want to provide efficient routing for resource restricted devices
|
||||
- We may want to provide compatibility with Waku v1 envelopes
|
||||
- We may want payloads to be encrypted by default
|
||||
- We may want to provide unlinkability for metadata protection
|
||||
|
||||
This specification attempts to provide for these various requirements.
|
||||
|
||||
# WakuMessage
|
||||
|
||||
A `WakuMessage` is what is being passed around by the other protocols, such as WakuRelay, WakuStore, and WakuFilter.
|
||||
|
||||
The `payload` field SHOULD contain whatever payload is being sent. See section below on payload encryption.
|
||||
|
||||
The `contentTopic` field SHOULD be filled out to allow for content-based filtering. See [Waku Filter spec](waku-filter.md) for details.
|
||||
|
||||
The `version` field MAY be filled out to allow for various types of payload encryption. Omitting it means the version is 0.
|
||||
|
||||
## Protobuf
|
||||
|
||||
```protobuf
|
||||
message WakuMessage {
|
||||
optional bytes payload = 1;
|
||||
optional uint32 contentTopic = 2;
|
||||
optional uint32 version = 3;
|
||||
}
|
||||
```
|
||||
|
||||
## Payload encryption
|
||||
|
||||
Payload encryption depends on the `version` field.
|
||||
|
||||
### Version 0
|
||||
|
||||
This indicates that the payload SHOULD be either unencrypted or that encryption is done at a separate layer outside of Waku.
|
||||
|
||||
### Version 1 (not yet implemented in Waku v2)
|
||||
|
||||
This indicates that payloads MUST be encrypted using [Waku v1 envelope data
|
||||
format spec](https://specs.vac.dev/specs/waku/v1/envelope-data-format.html).
|
||||
|
||||
This provides for asymmetric and symmetric encryption. Key agreement is out of band. It also provides an encrypted signature and padding for some form of unlinkability.
|
||||
|
||||
# Differences from Whisper / Waku v1 envelopes
|
||||
|
||||
In Whisper and Waku v1, an envelope contains the following fields: `expiry, ttl,
|
||||
topic, data, nonce`.
|
||||
|
||||
Since Waku v2 is using libp2p PubSub, some of these fields can be dropped. The previous `topic`
|
||||
field corresponds to `contentTopic`. The previous `data` field corresponds to the `payload` field.
|
||||
|
||||
# Security Consideration
|
||||
|
||||
In Waku, the confidentiality, integrity, and authenticity of the data must be addressed at the `WakuMessage` level. That is, the `payload` shall be encrypted or signed properly to meet the application-specific privacy needs.
|
||||
|
||||
# Changelog
|
||||
|
||||
Initial release on [2020-10-22](https://github.com/vacp2p/specs/pull/222/commits/dbab6c7084b414b62818150588266abedd09315f).
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,66 @@
|
|||
---
|
||||
slug: 15
|
||||
title: 15/WAKU-BRIDGE
|
||||
name: Waku Bridge
|
||||
status: raw
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
---
|
||||
|
||||
A bridge between Waku v1 and Waku v2.
|
||||
|
||||
# Bridge
|
||||
|
||||
A bridge requires supporting both Waku versions:
|
||||
|
||||
* Waku v1 - using devp2p RLPx protocol
|
||||
* Waku v2 - using libp2p protocols
|
||||
|
||||
Packets received on the Waku v1 network SHOULD be published just once on the
|
||||
Waku v2 network. More specifically, the bridge SHOULD publish
|
||||
this through the Waku Relay (PubSub domain).
|
||||
|
||||
Publishing such packet will require the creation of a new `Message` with a
|
||||
new `WakuMessage` as data field. The `data` and `topic` field from the Waku v1
|
||||
`Envelope` MUST be copied to the `payload` and `contentTopic` fields of the
|
||||
`WakuMessage`. Other fields such as nonce, expiry and ttl will be dropped as
|
||||
they become obsolete in Waku v2.
|
||||
|
||||
Before this is done, the usual envelope verification still applies:
|
||||
|
||||
* Expiry & future time verification
|
||||
* PoW verification
|
||||
* Size verification
|
||||
|
||||
Bridging SHOULD occur through the `WakuRelay`, but it MAY also be done on other Waku
|
||||
v2 protocols (e.g. `WakuFilter`). The latter is however not advised as it will
|
||||
increase the complexity of the bridge and because of the
|
||||
[Security Considerations](#security-considerations) explained further below.
|
||||
|
||||
Packets received on the Waku v2 network SHOULD be posted just once on the Waku
|
||||
v1 network. The Waku v2 `WakuMessage` contains only the `payload` and
|
||||
`contentTopic` fields. The bridge MUST create a new Waku v1 `Envelope` and
|
||||
copy over the `payload` and `contentFilter` fields to the `data` and `topic`
|
||||
fields. Next, before posting on the network, the bridge MUST set a new expiry
|
||||
and ttl and do the PoW nonce calculation.
|
||||
|
||||
### Security Considerations
|
||||
As mentioned above, a bridge will be posting new Waku v1 envelopes, which
|
||||
requires doing the PoW nonce calculation.
|
||||
|
||||
This could be a DoS attack vector, as the PoW calculation will make it more
|
||||
expensive to post the message compared to the original publishing on the Waku v2
|
||||
network. Low PoW setting will lower this problem, but it is likely that it is
|
||||
still more expensive.
|
||||
|
||||
For this reason, bridges SHOULD probably be run independently of other nodes, so
|
||||
that a bridge that gets overwhelmed does not disrupt regular Waku v2 to v2
|
||||
traffic.
|
||||
|
||||
Bridging functionality SHOULD also be carefully implemented so that messages do
|
||||
not bounce back and forth between the two networks. The bridge SHOULD properly
|
||||
track messages with a seen filter so that no amplification can be achieved here.
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via
|
||||
[CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,629 @@
|
|||
---
|
||||
slug: 16
|
||||
title: 16/WAKU2-RPC
|
||||
name: Waku v2 RPC API
|
||||
status: raw
|
||||
editor: Hanno Cornelius <hanno@status.im>
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
This specification describes the JSON-RPC API that Waku v2 nodes MAY adhere to. Refer to the [Waku v2 specification](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-v2.md) for more information on Waku v2.
|
||||
|
||||
# Wire Protocol
|
||||
|
||||
## Transport
|
||||
|
||||
Nodes SHOULD expose an accessible [JSON-RPC](https://www.jsonrpc.org/specification) API. The JSON-RPC version SHOULD be `2.0`. Below is an example request:
|
||||
|
||||
```json
|
||||
{
|
||||
"jsonrpc":"2.0",
|
||||
"method":"get_waku_v2_debug_info",
|
||||
"params":[],
|
||||
"id":1
|
||||
}
|
||||
```
|
||||
|
||||
### Fields
|
||||
|
||||
| Field | Description |
|
||||
| --------- | --------------------------------------------------- |
|
||||
| `jsonrpc` | Contains the used JSON-RPC version (`Default: 2.0`) |
|
||||
| `method` | Contains the JSON-RPC method that is being called |
|
||||
| `params` | An array of parameters for the request |
|
||||
| `id` | The request ID |
|
||||
|
||||
## Types
|
||||
|
||||
In this specification, the primitive types `Boolean`, `String`, `Number` and `Null`, as well as the structured types `Array` and `Object`, are to be interpreted according to the [JSON-RPC specification](https://www.jsonrpc.org/specification#conventions). It also adopts the same capitalisation conventions.
|
||||
|
||||
The following structured types are defined for use throughout the document:
|
||||
|
||||
### WakuMessage
|
||||
|
||||
Refer to [`Waku Message` specification](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-message.md) for more information.
|
||||
|
||||
`WakuMessage` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ---: | :---: | :---: | --- |
|
||||
| `payload` | `String` | mandatory | The message payload as a hex encoded data string |
|
||||
| `contentTopic` | `Number` | optional | Message content topic for optional content-based filtering |
|
||||
| `version` | `Number` | optional | Message version. Used to indicate type of payload encryption. Default version is 0 (no payload encryption). |
|
||||
|
||||
## Method naming
|
||||
|
||||
The JSON-RPC methods in this document are designed to be mappable to HTTP REST endpoints. Method names follow the pattern `<method_type>_waku_<protocol_version>_<api>_<api_version>_<resource>`
|
||||
|
||||
- `<method_type>`: prefix of the HTTP method type that most closely matches the JSON-RPC function. Supported `method_type` values are `get`, `post`, `put`, `delete` or `patch`.
|
||||
- `<protocol_version>`: Waku version. Currently **v2**.
|
||||
- `<api>`: one of the listed APIs below, e.g. `store`, `debug`, or `relay`.
|
||||
- `<api_version>`: API definition version. Currently **v1** for all APIs.
|
||||
- `<resource>`: the resource or resource path being addressed
|
||||
|
||||
The method `post_waku_v2_relay_v1_message`, for example, would map to the HTTP REST endpoint `POST /waku/v2/relay/v1/message`.
|
||||
|
||||
## Debug API
|
||||
|
||||
### Types
|
||||
|
||||
The following structured types are defined for use on the Debug API:
|
||||
|
||||
#### WakuInfo
|
||||
|
||||
`WakuInfo` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `listenStr` | `String` | mandatory | Address that the node is listening for |
|
||||
|
||||
### `get_waku_v2_debug_v1_info`
|
||||
|
||||
The `get_waku_v2_debug_v1_info` method retrieves information about a Waku v2 node
|
||||
|
||||
#### Parameters
|
||||
|
||||
none
|
||||
|
||||
#### Response
|
||||
- [**`WakuInfo`**](#WakuInfo) - information about a Waku v2 node
|
||||
|
||||
## Relay API
|
||||
|
||||
Refer to the [Waku Relay specification](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-relay.md) for more information on the relaying of messages.
|
||||
|
||||
### Types
|
||||
|
||||
The following structured types are defined for use on the Relay API:
|
||||
|
||||
#### WakuRelayMessage
|
||||
|
||||
`WakuRelayMessage` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: | ----------- |
|
||||
| `payload` | `String` | mandatory | The payload being relayed as a hex encoded data string |
|
||||
| `contentTopic` | `Number` | optional | Message content topic for optional content-based filtering |
|
||||
|
||||
> **_NOTE:_** `WakuRelayMessage` maps directly to a [`WakuMessage`](#WakuMessage), except that the latter contains an explicit message `version`. For `WakuRelay` purposes, the versioning is handled by the API.
|
||||
|
||||
### `post_waku_v2_relay_v1_message`
|
||||
|
||||
The `post_waku_v2_relay_v1_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor)
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
|
||||
| `message` | [`WakuRelayMessage`](#WakuRelayMessage) | mandatory | The `message` being relayed |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `post_waku_v2_relay_v1_subscriptions`
|
||||
|
||||
The `post_waku_v2_relay_v1_subscriptions` method subscribes a node to an array of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topics` | `Array`[`String`] | mandatory | The [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being subscribed to |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `delete_waku_v2_relay_v1_subscriptions`
|
||||
|
||||
The `delete_waku_v2_relay_v1_subscriptions` method unsubscribes a node from an array of [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topics` | `Array`[`String`] | mandatory | The [PubSub `topics`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being unsubscribed from |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `get_waku_v2_relay_v1_messages`
|
||||
|
||||
The `get_waku_v2_relay_v1_messages` method returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Array`[[`WakuMessage`](#WakuMessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
## Store API
|
||||
|
||||
Refer to the [Waku Store specification](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-store.md) for more information on message history retrieval.
|
||||
|
||||
### Types
|
||||
|
||||
The following structured types are defined for use on the Store API:
|
||||
|
||||
#### StoreResponse
|
||||
|
||||
`StoreResponse` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `messages` | `Array`[[`WakuMessage`](#WakuMessage)] | mandatory | Array of retrieved historical messages |
|
||||
| `pagingOptions` | [`PagingOptions`](#PagingOptions) | [conditional](#get_waku_v2_store_v1_messages) | Paging information from which to resume further historical queries |
|
||||
|
||||
|
||||
#### PagingOptions
|
||||
|
||||
`PagingOptions` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `pageSize` | `Number` | mandatory | Number of messages to retrieve per page |
|
||||
| `cursor` | [`Index`](#Index) | optional | Message [`Index`](#Index) from which to perform pagination. If not included and `forward` is set to `true`, paging will be performed from the beginning of the list. If not included and `forward` is set to `false`, paging will be performed from the end of the list.|
|
||||
| `forward` | `Bool` | mandatory | `true` if paging forward, `false` if paging backward |
|
||||
|
||||
#### Index
|
||||
|
||||
`Index` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `digest` | `String` | mandatory | A hash for the message at this [`Index`](#Index) |
|
||||
| `receivedTime` | `Number` | mandatory | UNIX timestamp at which the message at this [`Index`](#Index) was received |
|
||||
|
||||
### `get_waku_v2_store_v1_messages`
|
||||
|
||||
The `get_waku_v2_store_v1_messages` method retrieves historical messages on specific content topics. This method MAY be called with [`PagingOptions`](#PagingOptions), to retrieve historical messages on a per-page basis. If the request included [`PagingOptions`](#PagingOptions), the node MUST return messages on a per-page basis and include [`PagingOptions`](#PagingOptions) in the response. These [`PagingOptions`](#PagingOptions) MUST contain a `cursor` pointing to the [`Index`](#Index) from which a new page can be requested.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topics` | `Array`[`Number`] | mandatory | Array of content topics to query for historical messages |
|
||||
| `pagingOptions` | [`PagingOptions`](#PagingOptions) | optional | Pagination information |
|
||||
|
||||
#### Response
|
||||
|
||||
- [**`StoreResponse`**](#StoreResponse) - the response to a `query` for historical messages.
|
||||
|
||||
## Filter API
|
||||
|
||||
Refer to the [Waku Filter specification](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-filter.md) for more information on content filtering.
|
||||
|
||||
### Types
|
||||
|
||||
The following structured types are defined for use on the Filter API:
|
||||
|
||||
#### ContentFilter
|
||||
|
||||
`ContentFilter` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topics` | `Array`[`Number`] | mandatory | Array of message content topics |
|
||||
|
||||
### `post_waku_v2_filter_v1_subscription`
|
||||
|
||||
The `post_waku_v2_filter_v1_subscription` method creates a subscription in a [light node](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-filter.md#rationale) for messages that matches a content filter and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `contentFilters` | `Array`[[`ContentFilter`](#ContentFilter)] | mandatory | Array of content filters being subscribed to |
|
||||
| `topic` | `String` | optional | Message topic |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `delete_waku_v2_filter_v1_subscription`
|
||||
|
||||
The `delete_waku_v2_filter_v1_subscription` method removes subscriptions in a [light node](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-filter.md#rationale) matching a content filter and, optionally, a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `contentFilters` | `Array`[[`ContentFilter`](#ContentFilter)] | mandatory | Array of content filters being unsubscribed from |
|
||||
| `topic` | `String` | optional | Message topic |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `get_waku_v2_filter_v1_messages`
|
||||
|
||||
The `get_waku_v2_filter_v1_messages` method returns a list of messages that were received on a subscribed content `topic` after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled content `topic`. If no message has yet been received on the polled content `topic`, the server SHOULD respond with an empty list. This method can be used to poll a content `topic` for new messages.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `contentTopic` | `Number` | mandatory | The content topic to poll for the latest messages |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Array`[[`WakuMessage`](#WakuMessage)]** - the latest `messages` on the polled content `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
## Admin API
|
||||
|
||||
The Admin API provides privileged accesses to the internal operations of a Waku v2 node.
|
||||
|
||||
### Types
|
||||
|
||||
The following structured types are defined for use on the Admin API:
|
||||
|
||||
#### WakuPeer
|
||||
|
||||
`WakuPeer` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `multiaddr` | `String` | mandatory | Multiaddress containing this peer's location and identity |
|
||||
| `protocol` | `String` | mandatory | Protocol that this peer is registered for |
|
||||
| `connected` | `bool` | mandatory | `true` if peer has active connection for this `protocol`, `false` if not |
|
||||
|
||||
### `get_waku_v2_admin_v1_peers`
|
||||
|
||||
The `get_waku_v2_admin_v1_peers` method returns an array of peers registered on this node. Since a Waku v2 node may open either continuous or ad hoc connections, depending on the negotiated protocol, these peers may have different connected states. The same peer MAY appear twice in the returned array, if it is registered for more than one protocol.
|
||||
|
||||
#### Parameters
|
||||
|
||||
none
|
||||
|
||||
#### Response
|
||||
- **`Array`[[`WakuPeer`](#WakuPeer)]** - Array of peers registered on this node
|
||||
|
||||
### `post_waku_v2_admin_v1_peers`
|
||||
|
||||
The `post_waku_v2_admin_v1_peers` method connects a node to a list of peers.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `peers` | `Array`[`String`] | mandatory | Array of peer `multiaddrs` to connect to. Each `multiaddr` must contain the [location and identity addresses](https://docs.libp2p.io/concepts/addressing/) of a peer. |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
|
||||
## Private API
|
||||
|
||||
The Private API provides functionality to encrypt/decrypt `WakuMessage` payloads using either symmetric or asymmetric cryptography. This allows backwards compatibility with [Waku v1 nodes](https://github.com/vacp2p/specs/blob/master/specs/waku/v1/waku-1.md).
|
||||
It is the API client's responsibility to keep track of the keys used for encrypted communication. Since keys must be cached by the client and provided to the node to encrypt/decrypt payloads, a Private API SHOULD NOT be exposed on non-local or untrusted nodes.
|
||||
|
||||
### Types
|
||||
|
||||
The following structured types are defined for use on the Private API:
|
||||
|
||||
#### KeyPair
|
||||
|
||||
`KeyPair` is an `Object` containing the following fields:
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `privateKey` | `String` | mandatory | Private key as hex encoded data string |
|
||||
| `publicKey` | `String` | mandatory | Public key as hex encoded data string |
|
||||
|
||||
### `get_waku_v2_private_v1_symmetric_key`
|
||||
|
||||
Generates and returns a symmetric key that can be used for message encryption and decryption.
|
||||
|
||||
#### Parameters
|
||||
|
||||
none
|
||||
|
||||
#### Response
|
||||
- **`String`** - A new symmetric key as hex encoded data string
|
||||
|
||||
### `get_waku_v2_private_v1_asymmetric_keypair`
|
||||
|
||||
Generates and returns a public/private key pair that can be used for asymmetric message encryption and decryption.
|
||||
|
||||
#### Parameters
|
||||
|
||||
none
|
||||
|
||||
#### Response
|
||||
- **[`KeyPair`](#KeyPair)** - A new public/private key pair as hex encoded data strings
|
||||
|
||||
### `post_waku_v2_private_v1_symmetric_message`
|
||||
|
||||
The `post_waku_v2_private_v1_symmetric_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
|
||||
|
||||
Before being relayed, the message payload is encrypted using the supplied symmetric key. The client MUST provide a symmetric key.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
|
||||
| `message` | [`WakuRelayMessage`](#WakuRelayMessage) | mandatory | The (unencrypted) `message` being relayed |
|
||||
| `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload encryption. This field MUST be included if symmetric key cryptography is selected |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `post_waku_v2_private_v1_asymmetric_message`
|
||||
|
||||
The `post_waku_v2_private_v1_asymmetric_message` method publishes a message to be relayed on a [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor).
|
||||
|
||||
Before being relayed, the message payload is encrypted using the supplied public key. The client MUST provide a public key.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) being published on |
|
||||
| `message` | [`WakuRelayMessage`](#WakuRelayMessage) | mandatory | The (unencrypted) `message` being relayed |
|
||||
| `publicKey` | `String` | mandatory | The hex encoded public key to use for payload encryption. This field MUST be included if asymmetric key cryptography is selected |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Bool`** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `get_waku_v2_private_v1_symmetric_messages`
|
||||
|
||||
The `get_waku_v2_private_v1_symmetric_messages` method decrypts and returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
|
||||
|
||||
Before returning the messages, the server decrypts the message payloads using the supplied symmetric key. The client MUST provide a symmetric key.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
|
||||
| `symkey` | `String` | mandatory | The hex encoded symmetric key to use for payload decryption. This field MUST be included if symmetric key cryptography is selected |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Array`[[`WakuRelayMessage`](#WakuRelayMessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
### `get_waku_v2_private_v1_asymmetric_messages`
|
||||
|
||||
The `get_waku_v2_private_v1_asymmetric_messages` method decrypts and returns a list of messages that were received on a subscribed [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) after the last time this method was called. The server MUST respond with an [error](https://www.jsonrpc.org/specification#error_object) if no subscription exists for the polled `topic`. If no message has yet been received on the polled `topic`, the server SHOULD return an empty list. This method can be used to poll a `topic` for new messages.
|
||||
|
||||
Before returning the messages, the server decrypts the message payloads using the supplied private key. The client MUST provide a private key.
|
||||
|
||||
#### Parameters
|
||||
|
||||
| Field | Type | Inclusion | Description |
|
||||
| ----: | :---: | :---: |----------- |
|
||||
| `topic` | `String` | mandatory | The [PubSub `topic`](https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-topic-descriptor) to poll for the latest messages |
|
||||
| `privateKey` | `String` | mandatory | The hex encoded private key to use for payload decryption. This field MUST be included if asymmetric key cryptography is selected |
|
||||
|
||||
#### Response
|
||||
|
||||
- **`Array`[[`WakuRelayMessage`](#WakuRelayMessage)]** - the latest `messages` on the polled `topic` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
# Example usage
|
||||
|
||||
## Store API
|
||||
|
||||
### `get_waku_v2_store_v1_messages`
|
||||
|
||||
This method is part of the `store` API and the specific resources to retrieve are (historical) `messages`. The protocol (`waku`) is on `v2`, whereas the Store API definition is on `v1`.
|
||||
|
||||
1. `get` *all* the historical messages for content topic **1**; no paging required
|
||||
|
||||
#### Request
|
||||
|
||||
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[[1]]}' --header "Content-Type: application/json" http://localhost:8545```
|
||||
|
||||
```jsonrpc
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "id",
|
||||
"method": "get_waku_v2_store_v1_messages",
|
||||
"params": [
|
||||
[
|
||||
1
|
||||
]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```jsonrpc
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "id",
|
||||
"result": {
|
||||
"messages": [
|
||||
{
|
||||
"payload": [
|
||||
1
|
||||
],
|
||||
"contentTopic": 1,
|
||||
"version": 0
|
||||
},
|
||||
{
|
||||
"payload": [
|
||||
2
|
||||
],
|
||||
"contentTopic": 1,
|
||||
"version": 0
|
||||
},
|
||||
{
|
||||
"payload": [
|
||||
3
|
||||
],
|
||||
"contentTopic": 1,
|
||||
"version": 0
|
||||
}
|
||||
],
|
||||
"pagingInfo": null
|
||||
},
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
2. `get` a single page of historical messages for content topic **1**; 2 messages per page, backward direction. Since this is the initial query, no `cursor` is provided, so paging will be performed from the end of the list.
|
||||
|
||||
#### Request
|
||||
|
||||
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[[1],{"pageSize":2,"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545```
|
||||
|
||||
```jsonrpc
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "id",
|
||||
"method": "get_waku_v2_store_v1_messages",
|
||||
"params": [
|
||||
[
|
||||
1
|
||||
],
|
||||
{
|
||||
"pageSize": 2,
|
||||
"forward": false
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```jsonrpc
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "id",
|
||||
"result": {
|
||||
"messages": [
|
||||
{
|
||||
"payload": [
|
||||
2
|
||||
],
|
||||
"contentTopic": 1,
|
||||
"version": 0
|
||||
},
|
||||
{
|
||||
"payload": [
|
||||
3
|
||||
],
|
||||
"contentTopic": 1,
|
||||
"version": 0
|
||||
}
|
||||
],
|
||||
"pagingInfo": {
|
||||
"pageSize": 2,
|
||||
"cursor": {
|
||||
"digest": "abcdef",
|
||||
"receivedTime": 1605887187
|
||||
},
|
||||
"forward": false
|
||||
}
|
||||
},
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
3. `get` the next page of historical messages for content topic **1**, using the cursor received above; 2 messages per page, backward direction.
|
||||
|
||||
#### Request
|
||||
|
||||
```curl -d '{"jsonrpc":"2.0","id":"id","method":"get_waku_v2_store_v1_messages", "params":[[1],{"pageSize":2,"cursor":{"digest":"abcdef","receivedTime":1605887187.00},"forward":false}]}' --header "Content-Type: application/json" http://localhost:8545```
|
||||
|
||||
```jsonrpc
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "id",
|
||||
"method": "get_waku_v2_store_v1_messages",
|
||||
"params": [
|
||||
[
|
||||
1
|
||||
],
|
||||
{
|
||||
"pageSize": 2,
|
||||
"cursor": {
|
||||
"digest": "abcdef",
|
||||
"receivedTime": 1605887187
|
||||
},
|
||||
"forward": false
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```jsonrpc
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "id",
|
||||
"result": {
|
||||
"messages": [
|
||||
{
|
||||
"payload": [
|
||||
1
|
||||
],
|
||||
"contentTopic": 1,
|
||||
"version": 0
|
||||
}
|
||||
],
|
||||
"pagingInfo": {
|
||||
"pageSize": 2,
|
||||
"cursor": {
|
||||
"digest": "123abc",
|
||||
"receivedTime": 1605866187
|
||||
},
|
||||
"forward": false
|
||||
}
|
||||
},
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
# References
|
||||
|
||||
1. [JSON-RPC specification](https://www.jsonrpc.org/specification)
|
||||
1. [LibP2P Addressing](https://docs.libp2p.io/concepts/addressing/)
|
||||
1. [LibP2P PubSub specification - topic descriptor](https://github.com/libp2p/specs/tree/master/pubsub#the-topic-descriptor)
|
||||
1. [Waku v2 specification](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-v2.md)
|
||||
|
||||
# Changelog
|
||||
|
||||
TBD
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,130 @@
|
|||
---
|
||||
slug: 17
|
||||
title: 17/WAKU-RLN
|
||||
name: Waku v2 RLN Relay
|
||||
status: raw
|
||||
editor: Sanaz Taheri <sanaz@status.im>
|
||||
---
|
||||
|
||||
The current specification embodies the details of the spam-protected version of `relay` protocol empowered by Rate Limiting Nullifiers (RLN). More details on RLN can be found in [this spec]() (TODO: to link the spec).
|
||||
|
||||
The security objective is to control the number of PubSub messages that each peer can publish per epoch (where epoch is a system design parameter), regardless of the published topic.
|
||||
|
||||
|
||||
**Protocol identifier***: `/vac/waku/waku-rln-relay/2.0.0-alpha1`
|
||||
|
||||
# Motivation
|
||||
|
||||
In open p2p messaging networks, one big problem is spam-resistance. Existing solutions, such as Whisper’s proof of work, are insufficient, especially for heterogeneous nodes. Other reputation-based approaches might not be desirable, due to issues around arbitrary exclusion and privacy.
|
||||
|
||||
We augment the `relay` protocol with a novel, light, and effective spam prevention mechanism which also suits the resource-constrained nodes.
|
||||
|
||||
TODO: Fill in more
|
||||
|
||||
|
||||
# Flow
|
||||
## SetUp and Registration
|
||||
A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members (realized by a Merkle Tree). An overview of registration is illustrated in Figure 1.
|
||||
|
||||
For the registration, a peer creates a transaction that sends x (TODO to be specified) ETH to the contract. The peer who has the "private key" `sk` associated with that deposit would be able to withdraw x ETH by providing valid proof. Note that `sk` is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch.
|
||||
|
||||
TODO: the interaction with the contract is subject to change depending on the final implementation
|
||||
|
||||
Once registered, the peer obtains the root of the tree (after the registration of the current peer) i.e., `root` as well as the authenticity path `authPath`. A peer can prove her membership using the `authPath`.
|
||||
|
||||
|
||||
`sk` and `authPath` are secret data and MUST be permanently and locally stored by the peer.
|
||||
|
||||
TODO: To specify the details of protobuf messages for the interaction with the contract
|
||||
|
||||
<!-- diagram -->
|
||||
|
||||
<p align="center">
|
||||
<img src="../../../assets/rln-relay/rln-relay.png" />
|
||||
<br />
|
||||
Figure 1: Registration.
|
||||
</p>
|
||||
|
||||
TODO: the function calls in this figure as well as messages are subject to change
|
||||
|
||||
## Publishing
|
||||
|
||||
In order to publish at a given `epoch`, the publishing peer proceeds based on the regular relay protocol. However, in order to protect against spamming, each PubSub message must carry a `proofBundle`. At a high level, the `proofBundle` is a zero-knowledge proof (TODO: to clarify what a zero-knowledge proof means) signifying that the publishing peer is a registered member, and she has not exceeded the messaging rate at the given `epoch`.
|
||||
|
||||
The `proofBundle` is embedded inside the `data` field of the PubSub message, which, in the `relay` protocol, corresponds to the `WakuMessage`. More details on the `proofBundle`'s message fields are provided under the Protobuf section.
|
||||
|
||||
The proof generation relies on the knowledge of `sk` and `authPath` (that is why they should be permanently and privately stored by the owning peer). Further inputs to the proof generation are `root`, `epoch` and `payload||contentTopic` where `payload` and `contentTopic` come from the `WakuMessage` (TODO: the inputs of the proof generation may change). The proof generation results in the following data items which are included as part of the `ProofBundle`:
|
||||
1. `shareX`
|
||||
2. `shareY`
|
||||
3. `nullifier`
|
||||
4. `zkProof`
|
||||
|
||||
|
||||
The tuple of (`nullifier`, `shareX`, `ShareY`) can be seen as partial disclosure of peer's `sk` for the intended `epoch`. Given two such tuples with identical `nullifier` but distinct `shareX`, `ShareY` results in full disclosure of peer's `sk` and hence burning the associated deposit. Note that the `nullifier` is a deterministic value derived from `sk` and `epoch` therefore any two messages issued by the same peer (i.e., sing the same `sk`) for the same `epoch` are guaranteed to have identical `nullifier`s.
|
||||
|
||||
Note that the `authPath` of each peer depends on the current status of the registration tree (hence changes when new peers register). As such, it is recommended (and necessary for anonymity) that the publisher updates her `authPath` based on the latest status of the tree and attempts the proof using her updated `authPath`.
|
||||
|
||||
|
||||
## Routing
|
||||
|
||||
Upon the receipt of a PubSub message, the routing peer needs to extract and parse the `proofBundle` from the `data` field. If the `epoch` attached to the message has a non-reasonable gap (TODO: the gap should be defined) with the routing peer's current `epoch` then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs).
|
||||
Furthermore, the routing peers MUST check whether the `proofBundle` is valid and the message is not spam. If both checks are passed successfully, then the message is relayed. If `proofBundle` is invalid then the message is dropped. If spamming is detected, the publishing peer gets slashed. An overview of routing procedure is depicted in Figure 2.
|
||||
|
||||
### Spam Detection and Slashing
|
||||
In order to enable local spam detection and slashing, routing peers MUST record the `nullifier`, `shareX`, and `shareY` of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.
|
||||
1. The routing peer first verifies the `zkProof` and drops the message if not verified.
|
||||
2. Otherwise, it checks whether a message with an identical `nullifier` has already been relayed.
|
||||
1. If such message exists and its `shareX` and `shareY` components are different from the incoming message, then slashing takes place (if the `shareX` and `shareY` fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).
|
||||
2. If none found, then the message gets relayed.
|
||||
|
||||
An overview of slashing procedure is provided in Figure 2.
|
||||
|
||||
TODO: may shorten or delete the Spam detection and slashing process
|
||||
|
||||
TODO: may consider [validator functions](https://github.com/libp2p/specs/tree/master/pubsub#topic-validation) or [extended validators](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#extended-validators) for the spam detection
|
||||
|
||||
<p align="center">
|
||||
<img src="../../../assets/rln-relay/rln-message-verification.png" />
|
||||
<br />
|
||||
Figure 2: Publishing, Routing and Slashing workflow.
|
||||
</p>
|
||||
|
||||
TODO: the function calls in this figure as well as messages are subject to change
|
||||
|
||||
# Security Considerations
|
||||
|
||||
TODO: add discussion about the anonymity (e.g., the `StrictNoSign` policy)
|
||||
|
||||
TODO: discuss about the economic spam guarantees
|
||||
|
||||
-------
|
||||
|
||||
# Protobuf
|
||||
|
||||
```protobuf
|
||||
//TODO may include the pubsub message
|
||||
// TODO to reflect this change on WakuMessage spec once the PR gets mature
|
||||
message WakuMessage {
|
||||
optional bytes payload = 1;
|
||||
optional uint32 contentTopic = 2;
|
||||
optional uint32 version = 3;
|
||||
optional ProofBundle proofBundle = 4;
|
||||
}
|
||||
|
||||
|
||||
message ProofBundle {
|
||||
int64 epoch = 1; // indicating the intended epoch of the message
|
||||
// TODO shareX and shareY
|
||||
bytes nullifier = 2;
|
||||
bytes root = 3; // TODO may be removed and added as part of zkProof
|
||||
// TODO zkProof
|
||||
}
|
||||
// TODO ZKProof may be a separate message type
|
||||
// TODO the protobuf messages for communicating with the contract
|
||||
|
||||
```
|
||||
TODO: to describe ProofBundle message fields
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,146 @@
|
|||
---
|
||||
slug: 18
|
||||
title: 18/WAKU2-SWAP
|
||||
name: Waku SWAP Accounting
|
||||
status: raw
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
This specification outlines how we do accounting and settlement based on the provision and usage of resources, most immediately bandwidth usage and/or storing and retrieving of Waku message. This enables nodes to cooperate and efficiently share resources, and in the case of unequal nodes to settle the difference through a relaxed payment mechanism in the form of sending cheques.
|
||||
|
||||
**Protocol identifier***: `/vac/waku/swap/2.0.0-alpha2`
|
||||
|
||||
# Motivation
|
||||
|
||||
The Waku network makes up a service network, and some nodes provide a useful service to other nodes. We want to account for that, and when imbalances arise, settle this. The core of this approach has some theoretical backing in game theory, and variants of it have practically been proven to work in systems such as Bittorrent. The specific model use was developed by the Swarm project (previously part of Ethereum), and we re-use contracts that were written for this purpose.
|
||||
|
||||
By using a delayed payment mechanism in the form of cheques, a barter-like mechanism can arise, and nodes can decide on their own policy as opposed to be strictly tied to a specific payment scheme. Additionally, this delayed settlement eases requirements on the underlying network in terms of transaction speed or costs.
|
||||
|
||||
Theoretically, nodes providing and using resources over a long, indefinite, period of time can be seen as an iterated form of [Prisoner's Dilemma (PD)](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma). Specifically, and more intuitively, since we have a cost and benefit profile for each provision/usage (of Waku Message's, e.g.), and the pricing can be set such that mutual cooperation is incentivized, this can be analyzed as a form of donations game.
|
||||
|
||||
# Game Theory - Iterated prisoner's dilemma / donation game
|
||||
|
||||
What follows is a sketch of what the game looks like between two nodes. We can look at it as a special case of iterated prisoner's dilemma called a [Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game) where each node can cooperate with some benefit `b` at a personal cost `c`, where `b>c`.
|
||||
|
||||
From A's point of view:
|
||||
|
||||
A/B | Cooperate | Defect
|
||||
-----|----------|-------
|
||||
Cooperate | b-c | -c
|
||||
Defect | b | 0
|
||||
|
||||
What this means is that if A and B cooperates, A gets some benefit `b` minus a cost `c`. If A cooperates and B defects she only gets the cost, and if she defects and B cooperates A only gets the benefit. If both defect they get neither benefit nor cost.
|
||||
|
||||
The generalized form of PD is:
|
||||
|
||||
A/B | Cooperate | Defect
|
||||
-----|----------|-------
|
||||
Cooperate | R | S
|
||||
Defect | T | P
|
||||
|
||||
With R=reward, S=Sucker's payoff, T=temptation, P=punishment
|
||||
|
||||
And the following holds:
|
||||
|
||||
- `T>R>P>S`
|
||||
- `2R>T+S`
|
||||
|
||||
In our case, this means `b>b-c>0>-c` and `2(b-c)> b-c` which is trivially true.
|
||||
|
||||
As this is an iterated game with no clear finishing point in most circumstances, a tit-for-tat strategy is simple, elegant and functional. To be more theoretically precise, this also requires reasonable assumptions on error rate and discount parameter. This captures notions such as "does the perceived action reflect the intended action" and "how much do you value future (uncertain) actions compared to previous actions". See [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation) for more details. In specific circumstances, nodes can choose slightly different policies if there's a strong need for it. A policy is simply how a node chooses to act given a set of circumstances.
|
||||
|
||||
A tit-for-tat strategy basically means:
|
||||
- cooperate first (perform service/beneficial action to other node)
|
||||
- defect when node stops cooperating (disconnect and similar actions), i.e. when it stops performing according to set parameters re settlement
|
||||
- resume cooperation if other node does so
|
||||
|
||||
This can be complemented with node selection mechanisms.
|
||||
|
||||
# SWAP Accounting
|
||||
|
||||
See [Book of Swarm](https://swarm-gateways.net/bzz:/latest.bookofswarm.eth/the-book-of-swarm.pdf) section 3.2. on Peer-to-peer accounting etc., for more context and details.
|
||||
|
||||
This approach is based on communicating payment thresholds and sending cheques
|
||||
as indications of later payments. The first part is done with a handshake, the
|
||||
second done once payment threshold is hit.
|
||||
|
||||
TODO: Illustrate payment and disconnection thresholds
|
||||
|
||||
TODO: Elaborate on how accounting works with amount in the context of e.g. store
|
||||
|
||||
TODO: Illustrate flow
|
||||
|
||||
A cheque is best thought of as a promise to pay at a later date.
|
||||
|
||||
TODO Specify chequebook
|
||||
|
||||
## Accounting
|
||||
|
||||
Nodes perform their own accounting for each relevant peer based on some "volume"/bandwidth metric. For now we take this to mean the number of `WakuMessage`s exchanged.
|
||||
|
||||
Additionally, a price is attached to each unit. For now, this is is simple a "karma counter" and equal to 1 per message.
|
||||
|
||||
NOTE: This may later be complemented with other metrics, either as part of SWAP or more likely outside of it. For example, online time can be communicated and attested to as a form of enhanced quality of service to inform peer selection.
|
||||
|
||||
## Flow
|
||||
|
||||
Assuming we have two store nodes, one operating mostly as a client (A) and another as server (B).
|
||||
|
||||
1. Node A performs a handshake with B node. B node responds and both nodes communicate their payment threshold.
|
||||
2. Node A and B creates an accounting entry for the other peer, keep track of peer and current balance.
|
||||
3. Node A issues a `HistoryRequest`, and B responds with a `HistoryResponse`. Based on the number of WakuMessages in the response, both nodes update their accounting records.
|
||||
4. When payment threshold is reached, Node A sends over a cheque to reach a neutral balance. Settlement of this is currently out of scope, but would occur through a SWAP contract (to be specified).
|
||||
5. If disconnect threshold is reached, Node B disconnects Node A.
|
||||
|
||||
## Policy
|
||||
|
||||
- If a node reaches a disconnect threshold, which MUST be outside the payment threshold, it SHOULD disconnect the other peer. For the PoC, this behavior can be mocked.
|
||||
- If a node is within payment balance, the other node SHOULD stay connected to it.
|
||||
- If a node receives a valid Cheque it SHOULD update its internal accounting records.
|
||||
- If any node behaves badly, the other node is free to disconnect and pick another node. Peer rating is out of scope of this specification.
|
||||
|
||||
## Protobuf
|
||||
|
||||
```
|
||||
// TODO What information to include here? For now "payment" threshold
|
||||
message Handshake {
|
||||
bytes payment_threshold = 1;
|
||||
}
|
||||
|
||||
// TODO Signature?
|
||||
// Should probably be over the whole Cheque type
|
||||
message Cheque {
|
||||
bytes beneficiary = 1;
|
||||
// TODO epoch time or block time?
|
||||
uint32 date = 2;
|
||||
// TODO ERC20 extension?
|
||||
// For now karma counter
|
||||
uint32 amount = 3;
|
||||
}
|
||||
```
|
||||
|
||||
## Enhancements
|
||||
|
||||
1. Settlements. At a later stage, these cheques can be redeemed. For now they
|
||||
are better thought of as a form of Karma.
|
||||
|
||||
2. More information to make a decision. E.g. incorporate QoS metrics such as
|
||||
online time to inform peer selection (slightly orthogonal to this).
|
||||
|
||||
## References
|
||||
|
||||
1. [Prisoner's Dilemma](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma)
|
||||
|
||||
2. [Donation game](https://en.wikipedia.org/wiki/Prisoner%27s_dilemma#Special_case:_donation_game)
|
||||
|
||||
3. [Axelrod - Evolution of Cooperation (book)](https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation)
|
||||
|
||||
4. [Book of Swarm](https://swarm-gateways.net/bzz:/latest.bookofswarm.eth/the-book-of-swarm.pdf)
|
||||
|
||||
# Changelog
|
||||
|
||||
TBD.
|
||||
|
||||
# Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,150 @@
|
|||
---
|
||||
slug: 2
|
||||
title: 2/MVDS
|
||||
name: Minimum Viable Data Synchronization
|
||||
status: stable
|
||||
editor: Sanaz Taheri <sanaz@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
In this specification, we describe a minimum viable protocol for data synchronization inspired by the Bramble Synchronization Protocol[^1]. This protocol is designed to ensure reliable messaging between peers across an unreliable peer-to-peer (P2P) network where they may be unreachable or unresponsive.
|
||||
|
||||
We present a reference implementation[^2] including a simulation to demonstrate its performance.
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Description |
|
||||
|------------|-------------------------------------------------------------------------------------|
|
||||
| **Peer** | The other nodes that a node is connected to. |
|
||||
| **Record** | Defines a payload element of either the type `OFFER`, `REQUEST`, `MESSAGE` or `ACK` |
|
||||
| **Node** | Some process that is able to store data, do processing and communicate for MVDS. |
|
||||
|
||||
## Wire Protocol
|
||||
|
||||
### Secure Transport
|
||||
|
||||
This specification does not define anything related to the transport of packets. It is assumed that this is abstracted in such a way that any secure transport protocol could be easily implemented. Likewise, properties such as confidentiality, integrity, authenticity and forward secrecy are assumed to be provided by a layer below.
|
||||
|
||||
### Payloads
|
||||
|
||||
Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
package vac.mvds;
|
||||
|
||||
message Payload {
|
||||
repeated bytes acks = 1;
|
||||
repeated bytes offers = 2;
|
||||
repeated bytes requests = 3;
|
||||
repeated Message messages = 4;
|
||||
}
|
||||
|
||||
message Message {
|
||||
bytes group_id = 1;
|
||||
int64 timestamp = 2;
|
||||
bytes body = 3;
|
||||
}
|
||||
```
|
||||
|
||||
*The payload field numbers are kept more "unique" to ensure no overlap with other protocol buffers.*
|
||||
|
||||
Each payload contains the following fields:
|
||||
|
||||
- **Acks:** This field contains a list (can be empty) of `message identifiers` informing the recipient that sender holds a specific message.
|
||||
- **Offers:** This field contains a list (can be empty) of `message identifiers` that the sender would like to give to the recipient.
|
||||
- **Requests:** This field contains a list (can be empty) of `message identifiers` that the sender would like to receive from the recipient.
|
||||
- **Messages:** This field contains a list of messages (can be empty).
|
||||
|
||||
**Message Identifiers:** Each `message` has a message identifier calculated by hashing the `group_id`, `timestamp` and `body` fields as follows:
|
||||
|
||||
```
|
||||
HASH("MESSAGE_ID", group_id, timestamp, body);
|
||||
```
|
||||
|
||||
**Group Identifiers:** Each `message` is assigned into a **group** using the `group_id` field, groups are independent synchronization contexts between peers.
|
||||
|
||||
The current `HASH` function used is `sha256`.
|
||||
|
||||
## Synchronization
|
||||
|
||||
### State
|
||||
|
||||
We refer to `state` as set of records for the types `OFFER`, `REQUEST` and `MESSAGE` that every node SHOULD store per peer. `state` MUST NOT contain `ACK` records as we do not retransmit those periodically. The following information is stored for records:
|
||||
|
||||
- **Type** - Either `OFFER`, `REQUEST` or `MESSAGE`
|
||||
- **Send Count** - The amount of times a record has been sent to a peer.
|
||||
- **Send Epoch** - The next epoch at which a record can be sent to a peer.
|
||||
|
||||
### Flow
|
||||
|
||||
A maximum of one payload SHOULD be sent to peers per epoch, this payload contains all `ACK`, `OFFER`, `REQUEST` and `MESSAGE` records for the specific peer. Payloads are created every epoch, containing reactions to previously received records by peers or new records being sent out by nodes.
|
||||
|
||||
Nodes MAY have two modes with which they can send records: `BATCH` and `INTERACTIVE` mode. The following rules dictate how nodes construct payloads every epoch for any given peer for both modes.
|
||||
|
||||
> ***NOTE:** A node may send messages both in interactive and in batch mode.*
|
||||
|
||||
#### Interactive Mode
|
||||
|
||||
- A node initially offers a `MESSAGE` when attempting to send it to a peer. This means an `OFFER` is added to the next payload and state for the given peer.
|
||||
- When a node receives an `OFFER`, a `REQUEST` is added to the next payload and state for the given peer.
|
||||
- When a node receives a `REQUEST` for a previously sent `OFFER`, the `OFFER` is removed from the state and the corresponding `MESSAGE` is added to the next payload and state for the given peer.
|
||||
- When a node receives a `MESSAGE`, the `REQUEST` is removed from the state and an `ACK` is added to the next payload for the given peer.
|
||||
- When a node receives an `ACK`, the `MESSAGE` is removed from the state for the given peer.
|
||||
- All records that require retransmission are added to the payload, given `Send Epoch` has been reached.
|
||||
|
||||
<p align="center">
|
||||
<img src="../assets/mvds/interactive.png" />
|
||||
<br />
|
||||
Figure 1: Delivery without retransmissions in interactive mode.
|
||||
</p>
|
||||
|
||||
#### Batch Mode
|
||||
|
||||
1. When a node sends a `MESSAGE`, it is added to the next payload and the state for the given peer.
|
||||
2. When a node receives a `MESSAGE`, an `ACK` is added to the next payload for the corresponding peer.
|
||||
3. When a node receives an `ACK`, the `MESSAGE` is removed from the state for the given peer.
|
||||
4. All records that require retransmission are added to the payload, given `Send Epoch` has been reached.
|
||||
|
||||
<!-- diagram -->
|
||||
|
||||
<p align="center">
|
||||
<img src="../assets/mvds/batch.png" />
|
||||
<br />
|
||||
Figure 2: Delivery without retransmissions in batch mode.
|
||||
</p>
|
||||
|
||||
|
||||
> ***NOTE:** Batch mode is higher bandwidth whereas interactive mode is higher latency.*
|
||||
|
||||
<!-- Interactions with state, flow chart with retransmissions? -->
|
||||
|
||||
### Retransmission
|
||||
|
||||
The record of the type `Type` SHOULD be retransmitted every time `Send Epoch` is smaller than or equal to the current epoch.
|
||||
|
||||
`Send Epoch` and `Send Count` MUST be increased every time a record is retransmitted. Although no function is defined on how to increase `Send Epoch`, it SHOULD be exponentially increased until reaching an upper bound where it then goes back to a lower epoch in order to prevent a record's `Send Epoch`'s from becoming too large.
|
||||
|
||||
> ***NOTE:** We do not retransmission `ACK`s as we do not know when they have arrived, therefore we simply resend them every time we receive a `MESSAGE`.*
|
||||
|
||||
## Formal Specification
|
||||
|
||||
MVDS has been formally specified using TLA+: <https://github.com/vacp2p/formalities/tree/master/MVDS>.
|
||||
|
||||
## Acknowledgments
|
||||
- Preston van Loon
|
||||
- Greg Markou
|
||||
- Rene Nayman
|
||||
- Jacek Sieka
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## Footnotes
|
||||
|
||||
[^1]: akwizgran et al. [BSP](https://code.briarproject.org/briar/briar-spec/blob/master/protocols/BSP.md). Briar.
|
||||
[^2]: <https://github.com/vacp2p/mvds>
|
|
@ -0,0 +1,219 @@
|
|||
---
|
||||
slug: 3
|
||||
title: 3/REMOTE-LOG
|
||||
name: Remote log specification
|
||||
status: draft
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
---
|
||||
|
||||
A remote log is a replication of a local log. This means a node can read data that originally came from a node that is offline.
|
||||
|
||||
This specification is complemented by a proof of concept implementation[^1].
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Definition |
|
||||
| ----------- | -------------------------------------------------------------------------------------- |
|
||||
| CAS | Content-addressed storage. Stores data that can be addressed by its hash. |
|
||||
| NS | Name system. Associates mutable data to a name. |
|
||||
| Remote log | Replication of a local log at a different location. |
|
||||
|
||||
## Wire Protocol
|
||||
|
||||
### Secure Transport, storage, and name system
|
||||
|
||||
This specification does not define anything related to: secure transport,
|
||||
content addressed storage, or the name system. It is assumed these capabilities
|
||||
are abstracted away in such a way that any such protocol can easily be
|
||||
implemented.
|
||||
|
||||
<!-- TODO: Elaborate on properties required here. -->
|
||||
|
||||
### Payloads
|
||||
|
||||
Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).
|
||||
|
||||
**CAS service**:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
package vac.cas;
|
||||
|
||||
service CAS {
|
||||
rpc Add(Content) returns (Address) {}
|
||||
rpc Get(Address) returns (Content) {}
|
||||
}
|
||||
|
||||
message Address {
|
||||
bytes id = 1;
|
||||
}
|
||||
|
||||
message Content {
|
||||
bytes data = 1;
|
||||
}
|
||||
```
|
||||
|
||||
<!-- XXX/TODO: Can we get rid of the id/data complication and just use bytes? -->
|
||||
|
||||
**NS service**:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
package vac.cas;
|
||||
|
||||
service NS {
|
||||
rpc Update(NameUpdate) returns (Response) {}
|
||||
rpc Fetch(Query) returns (Content) {}
|
||||
}
|
||||
|
||||
message NameUpdate {
|
||||
string name = 1;
|
||||
bytes content = 2;
|
||||
}
|
||||
|
||||
message Query {
|
||||
string name = 1;
|
||||
}
|
||||
|
||||
message Content {
|
||||
bytes data = 1;
|
||||
}
|
||||
|
||||
message Response {
|
||||
bytes data = 1;
|
||||
}
|
||||
```
|
||||
|
||||
<!-- XXX: Response and data type a bit weird, Ok/Err enum? -->
|
||||
<!-- TODO: Do we want NameInit here? -->
|
||||
|
||||
**Remote log:**
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
|
||||
package vac.cas;
|
||||
|
||||
message RemoteLog {
|
||||
repeated Pair pair = 1;
|
||||
bytes tail = 2;
|
||||
|
||||
message Pair {
|
||||
bytes remoteHash = 1;
|
||||
bytes localHash = 2;
|
||||
bytes data = 3;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<!-- TODO: Better name for Pair, Mapping? -->
|
||||
|
||||
<!-- TODO: Consider making more useful in conjunction with metadata field. It makes sense to explicitly list what sequence a message is <local hash, remote hash, data, seqid> this way I can easily sync a messages prior or after a specific number. To enable this to be dynamic it might make sense to add page info so that I am aware which page I can find seqid on -->
|
||||
|
||||
## Synchronization
|
||||
|
||||
### Roles
|
||||
|
||||
There are four fundamental roles:
|
||||
|
||||
1. Alice
|
||||
2. Bob
|
||||
2. Name system (NS)
|
||||
3. Content-addressed storage (CAS)
|
||||
|
||||
The *remote log* protobuf is what is stored in the name system.
|
||||
|
||||
"Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.
|
||||
|
||||
<!-- TODO: Document random node as remote log -->
|
||||
<!-- TODO: Document how to find initial remote log (e.g. per sync contexts -->
|
||||
|
||||
### Flow
|
||||
|
||||
<!-- diagram -->
|
||||
|
||||
<p align="center">
|
||||
<img src="../assets/remotelog/remote-log.png" />
|
||||
<br />
|
||||
Figure 1: Remote log data synchronization.
|
||||
</p>
|
||||
|
||||
<!-- Document the flow wrt operations -->
|
||||
|
||||
### Remote log
|
||||
|
||||
The remote log lets receiving nodes know what data they are missing. Depending
|
||||
on the specific requirements and capabilities of the nodes and name system, the
|
||||
information can be referred to differently. We distinguish between three rough
|
||||
modes:
|
||||
|
||||
1. Fully replicated log
|
||||
2. Normal sized page with CAS mapping
|
||||
3. "Linked list" mode - minimally sized page with CAS mapping
|
||||
|
||||
**Data format:**
|
||||
|
||||
```
|
||||
| H1_3 | H2_3 |
|
||||
| H1_2 | H2_2 |
|
||||
| H1_1 | H2_1 |
|
||||
| ------------|
|
||||
| next_page |
|
||||
```
|
||||
|
||||
Here the upper section indicates a list of ordered pairs, and the lower section
|
||||
contains the address for the next page chunk. `H1` is the native hash function,
|
||||
and `H2` is the one used by the CAS. The numbers corresponds to the messages.
|
||||
|
||||
To indicate which CAS is used, a remote log SHOULD use a multiaddr.
|
||||
|
||||
**Embedded data:**
|
||||
|
||||
A remote log MAY also choose to embed the wire payloads that corresponds to the
|
||||
native hash. This bypasses the need for a dedicated CAS and additional
|
||||
round-trips, with a trade-off in bandwidth usage.
|
||||
|
||||
```
|
||||
| H1_3 | | C_3 |
|
||||
| H1_2 | | C_2 |
|
||||
| H1_1 | | C_1 |
|
||||
| -------------|
|
||||
| next_page |
|
||||
```
|
||||
|
||||
Here `C` stands for the content that would be stored at the CAS.
|
||||
|
||||
Both patterns can be used in parallel, e,g. by storing the last `k` messages
|
||||
directly and use CAS pointers for the rest. Together with the `next_page` page
|
||||
semantics, this gives users flexibility in terms of bandwidth and
|
||||
latency/indirection, all the way from a simple linked list to a fully replicated
|
||||
log. The latter is useful for things like backups on durable storage.
|
||||
|
||||
### Next page semantics
|
||||
|
||||
The pointer to the 'next page' is another remote log entry, at a previous point
|
||||
in time.
|
||||
|
||||
<!-- TODO: Determine requirement re overlapping, adjacent, and/or missing entries -->
|
||||
|
||||
<!-- TODO: Document message ordering append only requirements -->
|
||||
|
||||
### Interaction with MVDS
|
||||
|
||||
[vac.mvds.Message](/spec/2#payloads) payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
TBD.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## Footnotes
|
||||
|
||||
[^1]: <https://github.com/vacp2p/research/tree/master/remote_log>
|
|
@ -0,0 +1,80 @@
|
|||
---
|
||||
slug: 4
|
||||
title: 4/MVDS-META
|
||||
name: MVDS Metadata Field
|
||||
status: draft
|
||||
editor: Sanaz Taheri <sanaz@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
- Oskar Thorén <oskar@status.im>
|
||||
---
|
||||
|
||||
In this specification, we describe a method to construct message history that will aid the consistency guarantees of [2/MVDS](/spec/2). Additionally, we explain how data sync can be used for more lightweight messages that do not require full synchronization.
|
||||
|
||||
## Motivation
|
||||
|
||||
In order for more efficient synchronization of conversational messages, information should be provided allowing a node to more effectively synchronize the dependencies for any given message.
|
||||
|
||||
## Format
|
||||
|
||||
We introduce the metadata message which is used to convey information about a message and how it SHOULD be handled.
|
||||
|
||||
```protobuf
|
||||
package vac.mvds;
|
||||
|
||||
message Metadata {
|
||||
repeated bytes parents = 1;
|
||||
bool ephemeral = 2;
|
||||
}
|
||||
```
|
||||
|
||||
Nodes MAY transmit a `Metadata` message by extending the MVDS [message](/spec/2#payloads) with a `metadata` field.
|
||||
|
||||
```diff
|
||||
message Message {
|
||||
bytes group_id = 1;
|
||||
int64 timestamp = 2;
|
||||
bytes body = 3;
|
||||
+ Metadata metadata = 4;
|
||||
}
|
||||
```
|
||||
|
||||
### Fields
|
||||
|
||||
| Name | Description |
|
||||
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `parents` | list of parent [`message identifier`s](/spec/2#payloads) for the specific message. |
|
||||
| `ephemeral` | indicates whether a message is ephemeral or not. |
|
||||
|
||||
## Usage
|
||||
|
||||
### `parents`
|
||||
|
||||
This field contains a list of parent [`message identifier`s](/spec/2#payloads) for the specific message. It MUST NOT contain any messages as parent whose `ack` flag was set to `false`. This establishes a directed acyclic graph (DAG)[^1] of persistent messages.
|
||||
|
||||
Nodes MAY buffer messages until dependencies are satisfied for causal consistency[^2], they MAY also pass the messages straight away for eventual consistency[^3].
|
||||
|
||||
A parent is any message before a new message that a node is aware of that has no children.
|
||||
|
||||
The number of parents for a given message is bound by [0, N], where N is the number of nodes participating in the conversation, therefore the space requirements for the `parents` field is O(N).
|
||||
|
||||
If a message has no parents it is considered a root. There can be multiple roots, which might be disconnected, giving rise to multiple DAGs.
|
||||
|
||||
### `ephemeral`
|
||||
|
||||
When the `ephemeral` flag is set to `false`, a node MUST send an acknowledgment when they have received and processed a message. If it is set to `true`, it SHOULD NOT send any acknowledgment. The flag is `false` by default.
|
||||
|
||||
Nodes MAY decide to not persist ephemeral messages, however they MUST NOT be shared as part of the message history.
|
||||
|
||||
Nodes SHOULD send ephemeral messages in batch mode. As their delivery is not needed to be guaranteed.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
## Footnotes
|
||||
|
||||
[^1]: <https://en.wikipedia.org/wiki/Directed_acyclic_graph>
|
||||
[^2]: Jepsen. [Causal Consistency](https://jepsen.io/consistency/models/causal). Jepsen, LLC.
|
||||
[^3]: <https://en.wikipedia.org/wiki/Eventual_consistency>
|
|
@ -0,0 +1,555 @@
|
|||
---
|
||||
slug: 5
|
||||
title: 5/WAKU0
|
||||
name: Waku v0
|
||||
status: deprecated
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Kim De Mey <kimdemey@status.im>
|
||||
---
|
||||
|
||||
This specification describes the format of Waku messages within the ÐΞVp2p Wire Protocol. This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627). Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic messages (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
|
||||
|
||||
## Motivation
|
||||
|
||||
Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku messages in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Definition |
|
||||
| --------------- | ----------------------------------------------------- |
|
||||
| **Light node** | A Waku node that does not forward any messages. |
|
||||
| **Envelope** | Messages sent and received by Waku nodes. |
|
||||
| **Node** | Some process that is able to communicate for Waku. |
|
||||
|
||||
## Underlying Transports and Prerequisites
|
||||
|
||||
### Use of DevP2P
|
||||
|
||||
For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
|
||||
|
||||
### Gossip based routing
|
||||
|
||||
In Whisper, messages are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Messages are eligible for retransmission until their TTL expires. A node SHOULD relay messages to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a [mailserver](./mailserver.md) response. A node SHOULD NOT send a message to a peer that it has already sent before.
|
||||
|
||||
## Wire Specification
|
||||
|
||||
### Use of RLPx transport protocol
|
||||
|
||||
All Waku messages are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf) for more details on RLP.
|
||||
|
||||
Waku is a RLPx subprotocol called `waku` with version `0`. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of `waku`, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
|
||||
|
||||
### ABNF specification
|
||||
|
||||
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
|
||||
|
||||
```abnf
|
||||
; Packet codes 0 - 127 are reserved for Waku protocol
|
||||
packet-code = 1*3DIGIT
|
||||
|
||||
; rate limits
|
||||
limit-ip = 1*DIGIT
|
||||
limit-peerid = 1*DIGIT
|
||||
limit-topic = 1*DIGIT
|
||||
|
||||
rate-limits = "[" limit-ip limit-peerid limit-topic "]"
|
||||
|
||||
pow-requirement-key = 48
|
||||
bloom-filter-key = 49
|
||||
light-node-key = 50
|
||||
confirmations-enabled-key = 51
|
||||
rate-limits-key = 52
|
||||
topic-interest-key = 53
|
||||
|
||||
status-options = "["
|
||||
[ pow-requirement-key pow-requirement ]
|
||||
[ bloom-filter-key bloom-filter ]
|
||||
[ light-node-key light-node ]
|
||||
[ confirmations-enabled-key confirmations-enabled ]
|
||||
[ rate-limits-key rate-limits ]
|
||||
[ topic-interest-key topic-interest ]
|
||||
"]"
|
||||
|
||||
status = "[" version status-options "]"
|
||||
|
||||
status-update = status-options
|
||||
|
||||
; version is "an integer (as specified in RLP)"
|
||||
version = DIGIT
|
||||
|
||||
confirmations-enabled = BIT
|
||||
|
||||
light-node = BIT
|
||||
|
||||
; pow is "a single floating point value of PoW.
|
||||
; This value is the IEEE 754 binary representation
|
||||
; of a 64-bit floating point number.
|
||||
; Values of qNAN, sNAN, INF and -INF are not allowed.
|
||||
; Negative values are also not allowed."
|
||||
pow = 1*DIGIT "." 1*DIGIT
|
||||
pow-requirement = pow
|
||||
|
||||
; bloom filter is "a byte array"
|
||||
bloom-filter = *OCTET
|
||||
|
||||
waku-envelope = "[" expiry ttl topic data nonce "]"
|
||||
|
||||
; List of topics interested in
|
||||
topic-interest = "[" *10000topic "]"
|
||||
|
||||
; 4 bytes (UNIX time in seconds)
|
||||
expiry = 4OCTET
|
||||
|
||||
; 4 bytes (time-to-live in seconds)
|
||||
ttl = 4OCTET
|
||||
|
||||
; 4 bytes of arbitrary data
|
||||
topic = 4OCTET
|
||||
|
||||
; byte array of arbitrary size
|
||||
; (contains encrypted message)
|
||||
data = OCTET
|
||||
|
||||
; 8 bytes of arbitrary data
|
||||
; (used for PoW calculation)
|
||||
nonce = 8OCTET
|
||||
|
||||
messages = 1*waku-envelope
|
||||
|
||||
; mail server / client specific
|
||||
p2p-request = waku-envelope
|
||||
p2p-message = 1*waku-envelope
|
||||
|
||||
; packet-format needs to be paired with its
|
||||
; corresponding packet-format
|
||||
packet-format = "[" packet-code packet-format "]"
|
||||
|
||||
required-packet = 0 status /
|
||||
1 messages /
|
||||
22 status-update /
|
||||
|
||||
optional-packet = 126 p2p-request / 127 p2p-message
|
||||
|
||||
packet = "[" required-packet [ optional-packet ] "]"
|
||||
```
|
||||
|
||||
All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from `0x00`.
|
||||
|
||||
### Packet Codes
|
||||
|
||||
The message codes reserved for Waku protocol: 0 - 127.
|
||||
|
||||
Messages with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
|
||||
|
||||
The Waku sub-protocol MUST support the following packet codes:
|
||||
|
||||
| Name | Int Value |
|
||||
| -------------------------- | ------------- |
|
||||
| Status | 0 |
|
||||
| Messages | 1 |
|
||||
| Status Update | 22 |
|
||||
|
||||
The following message codes are optional, but they are reserved for specific purpose.
|
||||
|
||||
| Name | Int Value | Comment |
|
||||
|----------------------------|-----------|---------|
|
||||
| Batch Ack | 11 | |
|
||||
| Message Response | 12 | |
|
||||
| P2P Request | 126 | |
|
||||
| P2P Message | 127 | |
|
||||
|
||||
### Packet usage
|
||||
|
||||
#### Status
|
||||
|
||||
The Status message serves as a Waku handshake and peers MUST exchange this
|
||||
message upon connection. It MUST be sent after the RLPx handshake and prior to
|
||||
any other Waku messages.
|
||||
|
||||
A Waku node MUST await the Status message from a peer before engaging in other Waku protocol activity with that peer.
|
||||
When a node does not receive the Status message from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
|
||||
|
||||
Upon retrieval of the Status message, the node SHOULD validate the message
|
||||
received and validated the Status message. Note that its peer might not be in
|
||||
the same state.
|
||||
|
||||
When a node is receiving other Waku messages from a peer before a Status
|
||||
message is received, the node MUST ignore these messages and SHOULD disconnect from that peer. Status messages received after the handshake is completed MUST also be ignored.
|
||||
|
||||
The status message MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
|
||||
|
||||
#### Messages
|
||||
|
||||
This packet is used for sending the standard Waku envelopes.
|
||||
|
||||
#### Status Update
|
||||
|
||||
The Status Update message is used to communicate an update of the settings of the node.
|
||||
The format is the same as the Status message, all fields are optional.
|
||||
If none of the options are specified the message MUST be ignored and considered a noop.
|
||||
Fields that are omitted are considered unchanged, fields that haven't changed SHOULD not
|
||||
be transmitted.
|
||||
|
||||
**PoW Requirement update**
|
||||
|
||||
When PoW is updated, peers MUST NOT deliver the sender envelopes with PoW lower than specified in this message.
|
||||
|
||||
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by message size and TTL:
|
||||
|
||||
PoW = (2**BestBit) / (size * TTL)
|
||||
|
||||
PoW calculation:
|
||||
|
||||
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
|
||||
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
|
||||
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
|
||||
|
||||
where size is the size of the RLP-encoded envelope, excluding `env_nonce` field (size of `short_rlp(envelope)`).
|
||||
|
||||
**Bloom filter update**
|
||||
|
||||
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
|
||||
|
||||
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
|
||||
|
||||
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
|
||||
|
||||
LET D[*] = 0
|
||||
FOREACH i IN { 0, 1, 2 } DO
|
||||
LET n = S[i]
|
||||
IF S[3] & (2 ** i) THEN n += 256
|
||||
D[n] = 1
|
||||
END FOR
|
||||
|
||||
A full bloom filter (all the bits set to 1) means that the node is to be considered a `Full Node` and it will accept any topic.
|
||||
|
||||
If both Topic Interest and bloom filter are specified, Topic Interest always takes precedence and bloom filter MUST be ignored.
|
||||
|
||||
If only bloom filter is specified, the current Topic Interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
|
||||
|
||||
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
|
||||
|
||||
**Topic Interest update**
|
||||
|
||||
This packet is used by Waku nodes for sharing their interest in messages with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
|
||||
|
||||
|
||||
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
|
||||
|
||||
If only Topic Interest is specified, the current bloom filter MUST be discarded and only the updated Topic Interest MUST be used when forwarding or posting envelopes.
|
||||
|
||||
An empty array signals that the node is not currently interested in receiving any envelope.
|
||||
|
||||
**Rate Limits update**
|
||||
|
||||
This packet is used for informing other nodes of their self defined rate limits.
|
||||
|
||||
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
|
||||
|
||||
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
|
||||
|
||||
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
|
||||
|
||||
Each node SHOULD broadcast its rate limits to its peers using the rate limits packet. The rate limits MAY also be sent as an optional parameter in the handshake.
|
||||
|
||||
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
|
||||
|
||||
**Message Confirmations update**
|
||||
|
||||
Message confirmations tell a node that a message originating from it has been received by its peers, allowing a node to know whether a message has or has not been received.
|
||||
|
||||
A node MAY send a message confirmation for any batch of messages received with a packet Messages Code.
|
||||
|
||||
A message confirmation is sent using Batch Acknowledge packet or Message Response packet. The Batch Acknowledge packet is followed by a keccak256 hash of the envelopes batch data.
|
||||
|
||||
The current `version` of the message response is `1`.
|
||||
|
||||
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
|
||||
|
||||
```abnf
|
||||
; a version of the Message Response
|
||||
version = 1*DIGIT
|
||||
|
||||
; keccak256 hash of the envelopes batch data (raw bytes) for which the confirmation is sent
|
||||
hash = *OCTET
|
||||
|
||||
hasherror = *OCTET
|
||||
|
||||
; error code
|
||||
code = 1*DIGIT
|
||||
|
||||
; a descriptive error message
|
||||
description = *ALPHA
|
||||
|
||||
error = "[" hasherror code description "]"
|
||||
errors = *error
|
||||
|
||||
response = "[" hash errors "]"
|
||||
|
||||
confirmation = "[" version response "]"
|
||||
```
|
||||
|
||||
The supported codes:
|
||||
`1`: means time sync error which happens when an envelope is too old or created in the future (the root cause is no time sync between nodes).
|
||||
|
||||
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message, a corresponding confirmation is broadcast by one or more peers.
|
||||
|
||||
|
||||
#### P2P Request
|
||||
|
||||
This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting old messages from the [Waku Mail Server](./mailserver.md).
|
||||
|
||||
#### P2P Message
|
||||
|
||||
This packet is used for sending the peer-to-peer messages, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of old (expired) messages, which is otherwise not allowed.
|
||||
|
||||
|
||||
### Payload Encryption
|
||||
|
||||
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
|
||||
|
||||
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
|
||||
|
||||
### Packet code Rationale
|
||||
|
||||
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions. Packet code `0x02` and `0x03` were previously used in Whisper but are deprecated as of Waku v0.4
|
||||
|
||||
Packet code `0x22` is used to dynamically change the settings of a node.
|
||||
|
||||
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client. Without P2P messages it would be impossible to deliver the old messages, since they will be recognized as expired, and the peer will be disconnected for violating the Whisper protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
|
||||
|
||||
## Additional capabilities
|
||||
|
||||
Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
|
||||
|
||||
Additionally there is the capability of a mailserver which is documented in its on [specification](mailserver.md).
|
||||
|
||||
### Light node
|
||||
|
||||
The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
|
||||
|
||||
Light nodes MUST NOT forward any incoming messages, they MUST only send their own messages. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in messages being dropped between the two.
|
||||
|
||||
Light nodes are identified by the `light_node` value in the status message.
|
||||
|
||||
### Accounting for resources (experimental)
|
||||
|
||||
Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf), and works by doing pairwise accounting for resources.
|
||||
|
||||
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
|
||||
|
||||
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
|
||||
|
||||
| peer | sent | received |
|
||||
|-------|------|----------|
|
||||
| peer1 | 0 | 123 |
|
||||
| peer2 | 10 | 40 |
|
||||
|
||||
In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
|
||||
|
||||
## Upgradability and Compatibility
|
||||
|
||||
### General principles and policy
|
||||
|
||||
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
|
||||
|
||||
1. Waku aims to be compatible with previous and future versions.
|
||||
|
||||
2. In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
|
||||
|
||||
3. To achieve this, we employ the following two general strategies:
|
||||
- a) Accretion (including protocol negotiation) over changing data
|
||||
- b) When we want to change things, we give it a new name (for example, a version number).
|
||||
|
||||
Examples:
|
||||
|
||||
- We enable bridging between `shh/6` and `waku/0` until such a time as when we are ready to gracefully drop support for `shh/6` (1, 2, 3).
|
||||
- When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3).
|
||||
- To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3)
|
||||
- When we we want to provide a new set of messages that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3)
|
||||
|
||||
### Backwards Compatibility
|
||||
|
||||
Waku is a different subprotocol from Whisper so it isn't directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
|
||||
|
||||
### Waku-Whisper bridging
|
||||
|
||||
`waku/0` and `shh/6` are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
|
||||
|
||||
**Roles:**
|
||||
- Waku client A, only Waku capability
|
||||
- Whisper client B, only Whisper capability
|
||||
- WakuWhisper bridge C, both Waku and Whisper capability
|
||||
|
||||
**Flow:**
|
||||
1. A posts message; B posts message.
|
||||
2. C picks up message from A and B and relays them both to Waku and Whisper.
|
||||
3. A receives message on Waku; B on Whisper.
|
||||
|
||||
**Note**: This flow means if another bridge C1 is active, we might get duplicate relaying for a message between C1 and C2. I.e. Whisper(<>Waku<>Whisper)<>Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
|
||||
|
||||
### Forward Compatibility
|
||||
|
||||
It is desirable to have a strategy for maintaining forward compatibility between `waku/0` and future version of waku. Here we outline some concerns and strategy for this.
|
||||
|
||||
- **Connecting to nodes with multiple versions:** The way this SHOULD be accomplished in the future is by negotiating the versions of subprotocols, within the `hello` message nodes transmit their capabilities along with a version. As suggested in [EIP-8](https://eips.ethereum.org/EIPS/eip-8), if a node connects that has a higher version number for a specific capability, the node with a lower number SHOULD assume backwards compatibility. The node with the higher version will decide if compatibility can be assured between versions, if this is not the case it MUST disconnect.
|
||||
- **Adding new packet codes:** New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet.
|
||||
- **Adding new options in `status-options`:** New options can be added to the `status-options` association list in the `status` and `status-update` packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving `status-options` with unknown option keys.
|
||||
|
||||
## Appendix A: Security considerations
|
||||
|
||||
There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as [mailservers](./mailserver.md#security-considerations) can be found in their respective specifications.
|
||||
|
||||
### Scalability and UX
|
||||
|
||||
**Bandwidth usage:**
|
||||
|
||||
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
|
||||
|
||||
**Gossip-based routing:**
|
||||
|
||||
Use of gossip-based routing doesn't necessarily scale. It means each node can see a message multiple times, and having too many light nodes can cause propagation probability that is too low. See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
|
||||
|
||||
**Lack of incentives:**
|
||||
|
||||
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
|
||||
|
||||
### Privacy
|
||||
|
||||
**Light node privacy:**
|
||||
|
||||
The main privacy concern with light nodes is that directly connected peers will know that a message originates from them (as it are the only ones it sends). This means nodes can make assumptions about what messages (topics) their peers are interested in.
|
||||
|
||||
**Bloom filter privacy:**
|
||||
|
||||
By having a bloom filter where only the topics you are interested in are set, you reveal which messages you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
|
||||
|
||||
**Privacy guarantees not rigorous:**
|
||||
|
||||
Privacy for Whisper / Waku haven't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
|
||||
|
||||
**Topic hygiene:**
|
||||
|
||||
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
|
||||
|
||||
### Spam resistance
|
||||
|
||||
**PoW bad for heterogeneous devices:**
|
||||
|
||||
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
|
||||
|
||||
### Censorship resistance
|
||||
|
||||
**Devp2p TCP port blockable:**
|
||||
|
||||
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
|
||||
|
||||
## Appendix B: Implementation Notes
|
||||
|
||||
### Implementation Matrix
|
||||
|
||||
| Client | Spec supported | Details |
|
||||
|--------|----------------|---------|
|
||||
| **Status-go** | 0.5 | [details](https://github.com/status-im/status-go/blob/develop/WAKU.md) |
|
||||
| **Nimbus** | 0.4 | [details](https://github.com/status-im/nimbus/tree/8747fe1ecd36fe778bb92b97634db84d364fede8/waku) |
|
||||
|
||||
### Recommendations for clients
|
||||
|
||||
Notes useful for implementing Waku mode.
|
||||
|
||||
1. Avoid duplicate envelopes
|
||||
|
||||
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
|
||||
|
||||
2. Topic specific recommendations
|
||||
|
||||
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
|
||||
|
||||
### Node discovery
|
||||
|
||||
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
|
||||
|
||||
Known static nodes MAY also be used.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 0.6
|
||||
|
||||
Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f24179844857520c68fa3e8f6018b125)
|
||||
|
||||
- Mark spec as Deprecated mode in terms of its lifecycle.
|
||||
|
||||
### Version 0.5
|
||||
|
||||
Released [March 17,2020](https://github.com/vacp2p/specs/commit/7b9dc562bc50c6bb844ac575cb221ec9cda2530a)
|
||||
|
||||
- Clarify the preferred way of handling unknown keys in the `status-options` association list.
|
||||
- Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior
|
||||
|
||||
### Version 0.4
|
||||
|
||||
Released [February 21, 2020](https://github.com/vacp2p/specs/commit/17bd066e317bbe33af07146b721d73f24de47e88).
|
||||
|
||||
- Simplify implementation matrix with latest state
|
||||
- Introduces a new required packet code Status Code (`0x22`) for communicating option changes
|
||||
- Deprecates the following packet codes: PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
|
||||
- Increased `topic-interest` capacity from 1000 to 10000
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [February 13, 2020](https://github.com/vacp2p/specs/commit/73138d6ba954ab4c315e1b8d210ac7631b6d1428).
|
||||
|
||||
- Recommend DNS based node discovery over other Discovery methods.
|
||||
- Mark spec as Draft mode in terms of its lifecycle.
|
||||
- Simplify Changelog and misc formatting.
|
||||
- Handshake/Status message not compatible with shh/6 nodes; specifying options as association list.
|
||||
- Include topic-interest in Status handshake.
|
||||
- Upgradability policy.
|
||||
- `topic-interest` packet code.
|
||||
|
||||
### Version 0.2
|
||||
|
||||
Released [December 10, 2019](https://github.com/vacp2p/specs/blob/waku-0.2.0/waku.md).
|
||||
|
||||
- General style improvements.
|
||||
- Fix ABNF grammar.
|
||||
- Mailserver requesting/receiving.
|
||||
- New packet codes: topic-interest (experimental), rate limits (experimental).
|
||||
- More details on handshake modifications.
|
||||
- Accounting for resources mode (experimental)
|
||||
- Appendix with security considerations: scalability and UX, privacy, and spam resistance.
|
||||
- Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
|
||||
- More details on handshake and parameters.
|
||||
- Describe rate limits in more detail.
|
||||
- More details on mailserver and mail client API.
|
||||
- Accounting for resources mode (very experimental).
|
||||
- Clarify differences with Whisper.
|
||||
|
||||
### Version 0.1
|
||||
|
||||
Initial version. Released [November 21, 2019](https://github.com/vacp2p/specs/blob/b59b9247f2ac1bf45c75bd3227a2e5dd87b6d7b0/waku.md).
|
||||
|
||||
### Differences between shh/6 and waku/0
|
||||
|
||||
Summary of main differences between this spec and Whisper v6, as described in [EIP-627](https://eips.ethereum.org/EIPS/eip-627):
|
||||
|
||||
- RLPx subprotocol is changed from `shh/6` to `waku/0`.
|
||||
- Light node capability is added.
|
||||
- Optional rate limiting is added.
|
||||
- Status packet has following additional parameters: light-node,
|
||||
confirmations-enabled and rate-limits
|
||||
- Mail Server and Mail Client functionality is now part of the specification.
|
||||
- P2P Message packet contains a list of envelopes instead of a single envelope.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
|
||||
## Footnotes
|
||||
|
||||
[^1]: Felix Lange et al. [The RLPx Transport Protocol](https://github.com/ethereum/devp2p/blob/master/rlpx.md). Ethereum.
|
|
@ -0,0 +1,627 @@
|
|||
---
|
||||
slug: 6
|
||||
title: 6/WAKU1
|
||||
name: Waku v1
|
||||
status: stable
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Andrea Maria Piana <andreap@status.im>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Kim De Mey <kimdemey@status.im>
|
||||
---
|
||||
|
||||
This specification describes the format of Waku packets within the ÐΞVp2p Wire Protocol. This spec substitutes [EIP-627](https://eips.ethereum.org/EIPS/eip-627). Waku is a fork of the original Whisper protocol that enables better usability for resource restricted devices, such as mostly-offline bandwidth-constrained smartphones. It does this through (a) light node support, (b) historic envelopes (with a mailserver) (c) expressing topic interest for better bandwidth usage and (d) basic rate limiting.
|
||||
|
||||
## Motivation
|
||||
|
||||
Waku was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices. We specify the standard for Waku packets in order to ensure forward compatibility of different Waku clients, backwards compatibility with Whisper clients, as well as to allow multiple implementations of Waku and its capabilities. We also modify the language to be more unambiguous, concise and consistent.
|
||||
|
||||
## Definitions
|
||||
|
||||
| Term | Definition |
|
||||
| --------------- | ----------------------------------------------------------------------------------------|
|
||||
| **Batch Ack** | An abbreviated term for Batch Acknowledgment |
|
||||
| **Light node** | A Waku node that does not forward any envelopes through the Messages packet. |
|
||||
| **Envelope** | Messages sent and received by Waku nodes. Described in [ABNF spec `waku-envelope`](#abnf-specification) |
|
||||
| **Node** | Some process that is able to communicate for Waku. |
|
||||
|
||||
## Underlying Transports and Prerequisites
|
||||
|
||||
### Use of DevP2P
|
||||
|
||||
For nodes to communicate, they MUST implement devp2p and run RLPx. They MUST have some way of connecting to other nodes. Node discovery is largely out of scope for this spec, but see the appendix for some suggestions on how to do this.
|
||||
|
||||
This protocol needs to advertise the `waku/1` [capability](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
|
||||
|
||||
### Gossip based routing
|
||||
|
||||
In Whisper, envelopes are gossiped between peers. Whisper is a form of rumor-mongering protocol that works by flooding to its connected peers based on some factors. Envelopes are eligible for retransmission until their TTL expires. A node SHOULD relay envelopes to all connected nodes if an envelope matches their PoW and bloom filter settings. If a node works in light mode, it MAY choose not to forward envelopes. A node MUST NOT send expired envelopes, unless the envelopes are sent as a [mailserver](spec/8) response. A node SHOULD NOT send an envelope to a peer that it has already sent before.
|
||||
|
||||
### Maximum Packet Size
|
||||
|
||||
Nodes SHOULD limit the maximum size of both packets and envelopes. If a packet or envelope exceeds its limit, it MUST be dropped.
|
||||
|
||||
- **RLPx Packet Size** - This size MUST be checked before a message is decoded.
|
||||
- **Waku Envelope Size** - Each envelope contained in an RLPx packet MUST then separately be checked against the maximum envelope size.
|
||||
|
||||
Clients MAY use their own maximum packet and envelope sizes. The default values are `1.5mb` for the RLPx Packet and `1mb` for a Waku envelope.
|
||||
|
||||
## Wire Specification
|
||||
|
||||
### Use of RLPx transport protocol
|
||||
|
||||
All Waku packets are sent as devp2p RLPx transport protocol, version 5[^1] packets. These packets MUST be RLP-encoded arrays of data containing two objects: packet code followed by another object (whose type depends on the packet code). See [informal RLP spec](https://github.com/ethereum/wiki/wiki/RLP) and the [Ethereum Yellow Paper, appendix B](https://ethereum.github.io/yellowpaper/paper.pdf) for more details on RLP.
|
||||
|
||||
Waku is a RLPx subprotocol called `waku` with version `0`. The version number corresponds to the major version in the header spec. Minor versions should not break compatibility of `waku`, this would result in a new major. (Some exceptions to this apply in the Draft stage of where client implementation is rapidly change).
|
||||
|
||||
### ABNF specification
|
||||
|
||||
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
|
||||
|
||||
```abnf
|
||||
; Packet codes 0 - 127 are reserved for Waku protocol
|
||||
packet-code = 1*3DIGIT
|
||||
|
||||
; rate limits per packet
|
||||
packet-limit-ip = 1*DIGIT
|
||||
packet-limit-peerid = 1*DIGIT
|
||||
packet-limit-topic = 1*DIGIT
|
||||
|
||||
; rate limits by size in bytes
|
||||
bytes-limit-ip = 1*DIGIT
|
||||
bytes-limit-peerid = 1*DIGIT
|
||||
bytes-limit-topic = 1*DIGIT
|
||||
|
||||
packet-rate-limits = "[" packet-limit-ip packet-limit-peerid packet-limit-topic "]"
|
||||
bytes-rate-limits = "[" bytes-limit-ip bytes-limit-peerid bytes-limit-topic "]"
|
||||
|
||||
pow-requirement-key = 0
|
||||
bloom-filter-key = 1
|
||||
light-node-key = 2
|
||||
confirmations-enabled-key = 3
|
||||
packet-rate-limits-key = 4
|
||||
topic-interest-key = 5
|
||||
bytes-rate-limits-key = 6
|
||||
|
||||
status-options = "["
|
||||
[ pow-requirement-key pow-requirement ]
|
||||
[ bloom-filter-key bloom-filter ]
|
||||
[ light-node-key light-node ]
|
||||
[ confirmations-enabled-key confirmations-enabled ]
|
||||
[ packet-rate-limits-key packet-rate-limits ]
|
||||
[ topic-interest-key topic-interest ]
|
||||
[ bytes-limits-key bytes-rate-limits ]
|
||||
"]"
|
||||
|
||||
status = status-options
|
||||
|
||||
status-update = status-options
|
||||
|
||||
confirmations-enabled = BIT
|
||||
|
||||
light-node = BIT
|
||||
|
||||
; pow is "a single floating point value of PoW.
|
||||
; This value is the IEEE 754 binary representation
|
||||
; of a 64-bit floating point number packed as a uint64.
|
||||
; Values of qNAN, sNAN, INF and -INF are not allowed.
|
||||
; Negative values are also not allowed."
|
||||
pow = 1*DIGIT "." 1*DIGIT
|
||||
pow-requirement = pow
|
||||
|
||||
; bloom filter is "a byte array"
|
||||
bloom-filter = *OCTET
|
||||
|
||||
waku-envelope = "[" expiry ttl topic data nonce "]"
|
||||
|
||||
; List of topics interested in
|
||||
topic-interest = "[" *10000topic "]"
|
||||
|
||||
; 4 bytes (UNIX time in seconds)
|
||||
expiry = 4OCTET
|
||||
|
||||
; 4 bytes (time-to-live in seconds)
|
||||
ttl = 4OCTET
|
||||
|
||||
; 4 bytes of arbitrary data
|
||||
topic = 4OCTET
|
||||
|
||||
; byte array of arbitrary size
|
||||
; (contains encrypted payload)
|
||||
data = *OCTET
|
||||
|
||||
; 8 bytes of arbitrary data
|
||||
; (used for PoW calculation)
|
||||
nonce = 8OCTET
|
||||
|
||||
messages = 1*waku-envelope
|
||||
|
||||
; version of the confirmation packet
|
||||
version = 1*DIGIT
|
||||
|
||||
; keccak256 hash of the envelopes batch data (raw bytes)
|
||||
; for which the confirmation is sent
|
||||
hash = *OCTET
|
||||
|
||||
hasherror = *OCTET
|
||||
|
||||
; error code
|
||||
code = 1*DIGIT
|
||||
|
||||
; a descriptive error message
|
||||
description = *ALPHA
|
||||
|
||||
error = "[" hasherror code description "]"
|
||||
errors = *error
|
||||
|
||||
response = "[" hash errors "]"
|
||||
|
||||
confirmation = "[" version response "]"
|
||||
|
||||
; message confirmation packet types
|
||||
batch-ack = confirmation
|
||||
message-response = confirmation
|
||||
|
||||
; mail server / client specific
|
||||
p2p-request = waku-envelope
|
||||
p2p-message = 1*waku-envelope
|
||||
p2p-request-complete = *OCTET
|
||||
|
||||
; packet-format needs to be paired with its
|
||||
; corresponding packet-format
|
||||
packet-format = "[" packet-code packet-format "]"
|
||||
|
||||
required-packet = 0 status /
|
||||
1 messages /
|
||||
22 status-update /
|
||||
|
||||
optional-packet = 11 batch-ack /
|
||||
12 message-response /
|
||||
126 p2p-request-complete /
|
||||
126 p2p-request /
|
||||
127 p2p-message
|
||||
|
||||
packet = "[" required-packet [ optional-packet ] "]"
|
||||
```
|
||||
|
||||
All primitive types are RLP encoded. Note that, per RLP specification, integers are encoded starting from `0x00`.
|
||||
|
||||
### Packet Codes
|
||||
|
||||
The packet codes reserved for Waku protocol: 0 - 127.
|
||||
|
||||
Packets with unknown codes MUST be ignored without generating any error, for forward compatibility of future versions.
|
||||
|
||||
The Waku sub-protocol MUST support the following packet codes:
|
||||
|
||||
| Name | Int Value |
|
||||
| -------------------------- | ------------- |
|
||||
| Status | 0 |
|
||||
| Messages | 1 |
|
||||
| Status Update | 22 |
|
||||
|
||||
The following message codes are optional, but they are reserved for specific purpose.
|
||||
|
||||
| Name | Int Value | Comment |
|
||||
|----------------------------|-----------|---------|
|
||||
| Batch Ack | 11 | |
|
||||
| Message Response | 12 | |
|
||||
| P2P Request Complete | 125 | |
|
||||
| P2P Request | 126 | |
|
||||
| P2P Message | 127 | |
|
||||
|
||||
### Packet usage
|
||||
|
||||
#### Status
|
||||
|
||||
The Status packet serves as a Waku handshake and peers MUST exchange this
|
||||
packet upon connection. It MUST be sent after the RLPx handshake and prior to
|
||||
any other Waku packets.
|
||||
|
||||
A Waku node MUST await the Status packet from a peer before engaging in other Waku protocol activity with that peer.
|
||||
When a node does not receive the Status packet from a peer, before a configurable timeout, it SHOULD disconnect from that peer.
|
||||
|
||||
Upon retrieval of the Status packet, the node SHOULD validate the packet
|
||||
received and validated the Status packet. Note that its peer might not be in
|
||||
the same state.
|
||||
|
||||
When a node is receiving other Waku packets from a peer before a Status
|
||||
packet is received, the node MUST ignore these packets and SHOULD disconnect from that peer. Status packets received after the handshake is completed MUST also be ignored.
|
||||
|
||||
The Status packet MUST contain an association list containing various options. All options within this association list are OPTIONAL, ordering of the key-value pairs is not guaranteed and therefore MUST NOT be relied on. Unknown keys in the association list SHOULD be ignored.
|
||||
|
||||
#### Messages
|
||||
|
||||
This packet is used for sending the standard Waku envelopes.
|
||||
|
||||
#### Status Update
|
||||
|
||||
The Status Update packet is used to communicate an update of the settings of the node.
|
||||
The format is the same as the Status packet, all fields are optional.
|
||||
If none of the options are specified the packet MUST be ignored and considered a noop.
|
||||
Fields that are omitted are considered unchanged, fields that haven't changed SHOULD not
|
||||
be transmitted.
|
||||
|
||||
##### PoW Requirement Field
|
||||
|
||||
When PoW Requirement is updated, peers MUST NOT deliver envelopes with PoW lower than the PoW Requirement specified.
|
||||
|
||||
PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by envelope size and TTL:
|
||||
|
||||
PoW = (2**BestBit) / (size * TTL)
|
||||
|
||||
PoW calculation:
|
||||
|
||||
fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field.
|
||||
fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce)
|
||||
fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)
|
||||
|
||||
where size is the size of the RLP-encoded envelope, excluding `env_nonce` field (size of `short_rlp(envelope)`).
|
||||
|
||||
##### Bloom Filter Field
|
||||
|
||||
The bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits.
|
||||
|
||||
Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic.
|
||||
|
||||
The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits.
|
||||
|
||||
LET D[*] = 0
|
||||
FOREACH i IN { 0, 1, 2 } DO
|
||||
LET n = S[i]
|
||||
IF S[3] & (2 ** i) THEN n += 256
|
||||
D[n] = 1
|
||||
END FOR
|
||||
|
||||
A full bloom filter (all the bits set to 1) means that the node is to be considered a `Full Node` and it will accept any topic.
|
||||
|
||||
If both topic interest and bloom filter are specified, topic interest always takes precedence and bloom filter MUST be ignored.
|
||||
|
||||
If only bloom filter is specified, the current topic interest MUST be discarded and only the updated bloom filter MUST be used when forwarding or posting envelopes.
|
||||
|
||||
A bloom filter with all bits set to 0 signals that the node is not currently interested in receiving any envelope.
|
||||
|
||||
##### Topic Interest Field
|
||||
|
||||
Topic interest is used to share a node's interest in envelopes with specific topics. It does this in a more bandwidth considerate way, at the expense of some metadata protection. Peers MUST only send envelopes with specified topics.
|
||||
|
||||
|
||||
It is currently bounded to a maximum of 10000 topics. If you are interested in more topics than that, this is currently underspecified and likely requires updating it. The constant is subject to change.
|
||||
|
||||
If only topic interest is specified, the current bloom filter MUST be discarded and only the updated topic interest MUST be used when forwarding or posting envelopes.
|
||||
|
||||
An empty array signals that the node is not currently interested in receiving any envelope.
|
||||
|
||||
##### Rate Limits Field
|
||||
|
||||
Rate limits is used to inform other nodes of their self defined rate limits.
|
||||
|
||||
In order to provide basic Denial-of-Service attack protection, each node SHOULD define its own rate limits. The rate limits SHOULD be applied on IPs, peer IDs, and envelope topics.
|
||||
|
||||
Each node MAY decide to whitelist, i.e. do not rate limit, selected IPs or peer IDs.
|
||||
|
||||
If a peer exceeds node's rate limits, the connection between them MAY be dropped.
|
||||
|
||||
Each node SHOULD broadcast its rate limits to its peers using the `status-update` packet. The rate limits MAY also be sent as an optional parameter in the handshake.
|
||||
|
||||
Each node SHOULD respect rate limits advertised by its peers. The number of packets SHOULD be throttled in order not to exceed peer's rate limits. If the limit gets exceeded, the connection MAY be dropped by the peer.
|
||||
|
||||
Two rate limits strategies are applied:
|
||||
|
||||
1) Number of packets per second
|
||||
2) Size of packets (in bytes) per second
|
||||
|
||||
Both strategies SHOULD be applied per IP address, peer id and topic.
|
||||
|
||||
The size limit SHOULD be greater or equal than the maximum packet size.
|
||||
|
||||
##### Light Node Field
|
||||
|
||||
When the node's `light-node` field is set to true, the node SHOULD NOT forward Envelopes from its peers.
|
||||
|
||||
A node connected to a peer with the `light-node` field set to true MUST NOT depend on the peer for forwarding Envelopes.
|
||||
|
||||
##### Confirmations Enabled Field
|
||||
|
||||
When the node's `confirmations-enabled` field is set to true, the node SHOULD send [message confirmations](#batch-ack-and-message-response) to its peers.
|
||||
|
||||
#### Batch Ack and Message Response
|
||||
|
||||
Message confirmations tell a node that an envelope originating from it has been received by its peers, allowing a node to know whether an envelope has or has not been received.
|
||||
|
||||
A node MAY send a message confirmation for any batch of envelopes received with a Messages packet (`0x01`).
|
||||
|
||||
A message confirmation is sent using Batch Ack packet (`0x0B`) or Message Response packet (`0x0C`). The message confirmation is specified in the [ABNF specification](#abnf-specification).
|
||||
|
||||
The current `version` in the `confirmation` is `1`.
|
||||
|
||||
The supported error codes:
|
||||
- `1`: time sync error which happens when an envelope is too old or was created in the future (typically because of an unsynchronized clock of a node).
|
||||
|
||||
The drawback of sending message confirmations is that it increases the noise in the network because for each sent envelope, a corresponding confirmation is broadcast by one or more peers.
|
||||
|
||||
#### P2P Request
|
||||
|
||||
This packet is used for sending Dapp-level peer-to-peer requests, e.g. Waku Mail Client requesting historic (expired) envelopes from the [Waku Mail Server](./mailserver.md).
|
||||
|
||||
#### P2P Message
|
||||
|
||||
This packet is used for sending the peer-to-peer envelopes, which are not supposed to be forwarded any further. E.g. it might be used by the Waku Mail Server for delivery of historic (expired) envelopes, which is otherwise not allowed.
|
||||
|
||||
#### P2P Request Complete
|
||||
|
||||
This packet is used to indicate that all envelopes, requested earlier with a P2P Request packet (`0x7E`), have been sent via one or more P2P Message packets (`0x7F`).
|
||||
|
||||
The content of the packet is explained in the [Waku Mail Server](./mailserver.md) specification.
|
||||
|
||||
### Payload Encryption
|
||||
|
||||
Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key.
|
||||
|
||||
Symmetric encryption uses AES GCM algorithm with random 96-bit nonce.
|
||||
|
||||
### Packet code Rationale
|
||||
|
||||
Packet codes `0x00` and `0x01` are already used in all Waku / Whisper versions. Packet code `0x02` and `0x03` were previously used in Whisper but are deprecated as of Waku v0.4
|
||||
|
||||
Packet code `0x22` is used to dynamically change the settings of a node.
|
||||
|
||||
Packet codes `0x7E` and `0x7F` may be used to implement Waku Mail Server and Client. Without the P2P Message packet it would be impossible to deliver the historic envelopes, since they will be recognized as expired, and the peer will be disconnected for violating the Waku protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades.
|
||||
|
||||
## Additional capabilities
|
||||
|
||||
Waku supports multiple capabilities. These include light node, rate limiting and bridging of traffic. Here we list these capabilities, how they are identified, what properties they have and what invariants they must maintain.
|
||||
|
||||
Additionally there is the capability of a mailserver which is documented in its on [specification](mailserver.md).
|
||||
|
||||
### Light node
|
||||
|
||||
The rationale for light nodes is to allow for interaction with waku on resource restricted devices as bandwidth can often be an issue.
|
||||
|
||||
Light nodes MUST NOT forward any incoming envelopes, they MUST only send their own envelopes. When light nodes happen to connect to each other, they SHOULD disconnect. As this would result in envelopes being dropped between the two.
|
||||
|
||||
Light nodes are identified by the `light_node` value in the Status packet.
|
||||
|
||||
### Accounting for resources (experimental)
|
||||
|
||||
Nodes MAY implement accounting, keeping track of resource usage. It is heavily inspired by Swarm's [SWAP protocol](https://www.bokconsulting.com.au/wp-content/uploads/2016/09/tron-fischer-sw3.pdf), and works by doing pairwise accounting for resources.
|
||||
|
||||
Each node keeps track of resource usage with all other nodes. Whenever an envelope is received from a node that is expected (fits bloom filter or topic interest, is legal, etc) this is tracked.
|
||||
|
||||
Every epoch (say, every minute or every time an event happens) statistics SHOULD be aggregated and saved by the client:
|
||||
|
||||
| peer | sent | received |
|
||||
|-------|------|----------|
|
||||
| peer1 | 0 | 123 |
|
||||
| peer2 | 10 | 40 |
|
||||
|
||||
In later versions this will be amended by nodes communication thresholds, settlements and disconnect logic.
|
||||
|
||||
## Upgradability and Compatibility
|
||||
|
||||
### General principles and policy
|
||||
|
||||
The currently advertised capability is `waku/1`. This needs to be advertised in the `hello` `ÐΞVp2p` [packet](https://ethereum.gitbooks.io/frontier-guide/devp2p.html).
|
||||
If a node supports multiple versions of `waku`, those needs to be explicitly advertised. For example if both `waku/0` and `waku/1` are supported, both `waku/0` and `waku/1` MUST be advertised.
|
||||
|
||||
These are policies that guide how we make decisions when it comes to upgradability, compatibility, and extensibility:
|
||||
|
||||
1. Waku aims to be compatible with previous and future versions.
|
||||
|
||||
2. In cases where we want to break this compatibility, we do so gracefully and as a single decision point.
|
||||
|
||||
3. To achieve this, we employ the following two general strategies:
|
||||
- a) Accretion (including protocol negotiation) over changing data
|
||||
- b) When we want to change things, we give it a new name (for example, a version number).
|
||||
|
||||
Examples:
|
||||
|
||||
- We enable bridging between `shh/6` and `waku/1` until such a time as when we are ready to gracefully drop support for `shh/6` (1, 2, 3).
|
||||
- When we add parameter fields, we (currently) do so by accreting them in a list, so old clients can ignore new fields (dynamic list) and new clients can use new capabilities (1, 3).
|
||||
- To better support (2) and (3) in the future, we will likely release a new version that gives better support for open, growable maps (association lists or native map type) (3)
|
||||
- When we we want to provide a new set of packets that have different requirements, we do so under a new protocol version and employ protocol versioning. This is a form of accretion at a level above - it ensures a client can support both protocols at once and drop support for legacy versions gracefully. (1,2,3)
|
||||
|
||||
### Backwards Compatibility
|
||||
|
||||
Waku is a different subprotocol from Whisper so it isn't directly compatible. However, the data format is the same, so compatibility can be achieved by the use of a bridging mode as described below. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility.
|
||||
|
||||
### Waku-Whisper bridging
|
||||
|
||||
`waku/1` and `shh/6` are different DevP2P subprotocols, however they share the same data format making their envelopes compatible. This means we can bridge the protocols naively, this works as follows.
|
||||
|
||||
**Roles:**
|
||||
- Waku client A, only Waku capability
|
||||
- Whisper client B, only Whisper capability
|
||||
- WakuWhisper bridge C, both Waku and Whisper capability
|
||||
|
||||
**Flow:**
|
||||
1. A posts envelope; B posts envelope.
|
||||
2. C picks up envelope from A and B and relays them both to Waku and Whisper.
|
||||
3. A receives envelope on Waku; B on Whisper.
|
||||
|
||||
**Note**: This flow means if another bridge C1 is active, we might get duplicate relaying for a envelope between C1 and C2. I.e. Whisper(<>Waku<>Whisper)<>Waku, A-C1-C2-B. Theoretically this bridging chain can get as long as TTL permits.
|
||||
|
||||
### Forward Compatibility
|
||||
|
||||
It is desirable to have a strategy for maintaining forward compatibility between `waku/1` and future version of waku. Here we outline some concerns and strategy for this.
|
||||
|
||||
- **Connecting to nodes with multiple versions:** The way this SHOULD be accomplished is by negotiating the versions of subprotocols, within the `hello` packet nodes transmit their capabilities along with a version. The highest common version should then be used.
|
||||
- **Adding new packet codes:** New packet codes can be added easily due to the available packet codes. Unknown packet codes SHOULD be ignored. Upgrades that add new packet codes SHOULD implement some fallback mechanism if no response was received for nodes that do not yet understand this packet.
|
||||
- **Adding new options in `status-options`:** New options can be added to the `status-options` association list in the `status` and `status-update` packet as options are OPTIONAL and unknown option keys SHOULD be ignored. A node SHOULD NOT disconnect from a peer when receiving `status-options` with unknown option keys.
|
||||
|
||||
## Appendix A: Security considerations
|
||||
|
||||
There are several security considerations to take into account when running Waku. Chief among them are: scalability, DDoS-resistance and privacy. These also vary depending on what capabilities are used. The security considerations for extra capabilities such as [mailservers](./mailserver.md#security-considerations) can be found in their respective specifications.
|
||||
|
||||
### Scalability and UX
|
||||
|
||||
#### Bandwidth usage:
|
||||
|
||||
In version 0 of Waku, bandwidth usage is likely to be an issue. For more investigation into this, see the theoretical scaling model described [here](https://github.com/vacp2p/research/tree/dcc71f4779be832d3b5ece9c4e11f1f7ec24aac2/whisper_scalability).
|
||||
|
||||
#### Gossip-based routing:
|
||||
|
||||
Use of gossip-based routing doesn't necessarily scale. It means each node can see an envelope multiple times, and having too many light nodes can cause propagation probability that is too low. See [Whisper vs PSS](https://our.status.im/whisper-pss-comparison/) for more and a possible Kademlia based alternative.
|
||||
|
||||
#### Lack of incentives:
|
||||
|
||||
Waku currently lacks incentives to run nodes, which means node operators are more likely to create centralized choke points.
|
||||
|
||||
### Privacy
|
||||
|
||||
#### Light node privacy:
|
||||
|
||||
The main privacy concern with light nodes is that directly connected peers will know that an envelope originates from them (as it are the only ones it sends). This means nodes can make assumptions about what envelopes (topics) their peers are interested in.
|
||||
|
||||
#### Bloom filter privacy:
|
||||
|
||||
By having a bloom filter where only the topics you are interested in are set, you reveal which envelopes you are interested in. This is a fundamental tradeoff between bandwidth usage and privacy, though the tradeoff space is likely suboptimal in terms of the [Anonymity](https://eprint.iacr.org/2017/954.pdf) [trilemma](https://petsymposium.org/2019/files/hotpets/slides/coordination-helps-anonymity-slides.pdf).
|
||||
|
||||
#### Privacy guarantees not rigorous:
|
||||
|
||||
Privacy for Whisper / Waku haven't been studied rigorously for various threat models like global passive adversary, local active attacker, etc. This is unlike e.g. Tor and mixnets.
|
||||
|
||||
#### Topic hygiene:
|
||||
|
||||
Similar to bloom filter privacy, if you use a very specific topic you reveal more information. See scalability model linked above.
|
||||
|
||||
### Spam resistance
|
||||
|
||||
**PoW bad for heterogeneous devices:**
|
||||
|
||||
Proof of work is a poor spam prevention mechanism. A mobile device can only have a very low PoW in order not to use too much CPU / burn up its phone battery. This means someone can spin up a powerful node and overwhelm the network.
|
||||
|
||||
### Censorship resistance
|
||||
|
||||
**Devp2p TCP port blockable:**
|
||||
|
||||
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
|
||||
|
||||
## Appendix B: Implementation Notes
|
||||
|
||||
### Implementation Matrix
|
||||
|
||||
| Client | Spec supported | Details |
|
||||
|--------|----------------|---------|
|
||||
| **Status-go** | 0.5 | [details](https://github.com/status-im/status-go/blob/develop/WAKU.md) |
|
||||
| **Nim-waku** | 1.0 | [details](https://github.com/status-im/nim-waku/blob/master/README.md) |
|
||||
|
||||
### Recommendations for clients
|
||||
|
||||
Notes useful for implementing Waku mode.
|
||||
|
||||
1. Avoid duplicate envelopes
|
||||
|
||||
To avoid duplicate envelopes, only connect to one Waku node. Benign duplicate envelopes is an intrinsic property of Whisper which often leads to a N factor increase in traffic, where N is the number of peers you are connected to.
|
||||
|
||||
2. Topic specific recommendations
|
||||
|
||||
Consider partition topics based on some usage, to avoid too much traffic on a single topic.
|
||||
|
||||
### Node discovery
|
||||
|
||||
Resource restricted devices SHOULD use [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) to discover nodes.
|
||||
|
||||
Known static nodes MAY also be used.
|
||||
|
||||
## Changelog
|
||||
|
||||
### [Initial Release](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)
|
||||
|
||||
- Add section on P2P Request Complete packet and update packet code table.
|
||||
- Correct the header hierarchy for the status-options fields.
|
||||
- Consistent use of the words packet, message and envelope.
|
||||
- Added section on max packet size
|
||||
- Complete the ABNF specification and minor ABNF fixes.
|
||||
|
||||
### Version 1.1
|
||||
|
||||
Released [June 09, 2020](https://github.com/vacp2p/specs/commit/33b8d7304c9ebece90ea94e601f11080a8ac2c4d)
|
||||
|
||||
- Add rate limit per bytes
|
||||
|
||||
### Version 1.0
|
||||
|
||||
Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f24179844857520c68fa3e8f6018b125)
|
||||
|
||||
- Removed `version` from handshake
|
||||
- Changed `RLP` keys from 48,49.. to 0,1..
|
||||
- Upgraded to `waku/1`
|
||||
|
||||
### Version 0.6
|
||||
|
||||
Released [April 21,2020](https://github.com/vacp2p/specs/commit/9e650995f24179844857520c68fa3e8f6018b125)
|
||||
|
||||
- Mark spec as Deprecated mode in terms of its lifecycle.
|
||||
|
||||
### Version 0.5
|
||||
|
||||
Released [March 17,2020](https://github.com/vacp2p/specs/commit/7b9dc562bc50c6bb844ac575cb221ec9cda2530a)
|
||||
|
||||
- Clarify the preferred way of handling unknown keys in the `status-options` association list.
|
||||
- Correct spec/implementation mismatch: Change RLP keys to be the their int values in order to reflect production behavior
|
||||
|
||||
### Version 0.4
|
||||
|
||||
Released [February 21, 2020](https://github.com/vacp2p/specs/commit/17bd066e317bbe33af07146b721d73f24de47e88).
|
||||
|
||||
- Simplify implementation matrix with latest state
|
||||
- Introduces a new required packet code Status Code (`0x22`) for communicating option changes
|
||||
- Deprecates the following packet codes: PoW Requirement (`0x02`), Bloom Filter (`0x03`), Rate limits (`0x20`), Topic interest (`0x21`) - all superseded by the new Status Code (`0x22`)
|
||||
- Increased `topic-interest` capacity from 1000 to 10000
|
||||
|
||||
### Version 0.3
|
||||
|
||||
Released [February 13, 2020](https://github.com/vacp2p/specs/commit/73138d6ba954ab4c315e1b8d210ac7631b6d1428).
|
||||
|
||||
- Recommend DNS based node discovery over other Discovery methods.
|
||||
- Mark spec as Draft mode in terms of its lifecycle.
|
||||
- Simplify Changelog and misc formatting.
|
||||
- Handshake/Status packet not compatible with shh/6 nodes; specifying options as association list.
|
||||
- Include topic-interest in Status handshake.
|
||||
- Upgradability policy.
|
||||
- `topic-interest` packet code.
|
||||
|
||||
### Version 0.2
|
||||
|
||||
Released [December 10, 2019](https://github.com/vacp2p/specs/blob/waku-0.2.0/waku.md).
|
||||
|
||||
- General style improvements.
|
||||
- Fix ABNF grammar.
|
||||
- Mailserver requesting/receiving.
|
||||
- New packet codes: topic-interest (experimental), rate limits (experimental).
|
||||
- More details on handshake modifications.
|
||||
- Accounting for resources mode (experimental)
|
||||
- Appendix with security considerations: scalability and UX, privacy, and spam resistance.
|
||||
- Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
|
||||
- More details on handshake and parameters.
|
||||
- Describe rate limits in more detail.
|
||||
- More details on mailserver and mail client API.
|
||||
- Accounting for resources mode (very experimental).
|
||||
- Clarify differences with Whisper.
|
||||
|
||||
### Version 0.1
|
||||
|
||||
Initial version. Released [November 21, 2019](https://github.com/vacp2p/specs/blob/b59b9247f2ac1bf45c75bd3227a2e5dd87b6d7b0/waku.md).
|
||||
|
||||
### Differences between shh/6 and waku/1
|
||||
|
||||
Summary of main differences between this spec and Whisper v6, as described in [EIP-627](https://eips.ethereum.org/EIPS/eip-627):
|
||||
|
||||
- RLPx subprotocol is changed from `shh/6` to `waku/1`.
|
||||
- Light node capability is added.
|
||||
- Optional rate limiting is added.
|
||||
- Status packet has following additional parameters: light-node,
|
||||
confirmations-enabled and rate-limits
|
||||
- Mail Server and Mail Client functionality is now part of the specification.
|
||||
- P2P Message packet contains a list of envelopes instead of a single envelope.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
|
||||
## Footnotes
|
||||
|
||||
[^1]: Felix Lange et al. [The RLPx Transport Protocol](https://github.com/ethereum/devp2p/blob/master/rlpx.md). Ethereum.
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
slug: 7
|
||||
title: 7/WAKU-DATA
|
||||
name: Waku Envelope data field
|
||||
status: stable
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
- Kim De Mey <kimdemey@status.im>
|
||||
---
|
||||
|
||||
This specification describes the encryption, decryption and signing of the content in the [data field used in Waku](/spec/6/#abnf-specification).
|
||||
|
||||
## Specification
|
||||
|
||||
The `data` field is used within the `waku envelope`, the field MUST contain the encrypted payload of the envelope.
|
||||
|
||||
The fields that are concatenated and encrypted as part of the `data` field are:
|
||||
- flags
|
||||
- auxiliary field
|
||||
- payload
|
||||
- padding
|
||||
- signature
|
||||
|
||||
In case of symmetric encryption, a `salt` (a.k.a. AES Nonce, 12 bytes) field MUST be appended.
|
||||
|
||||
### ABNF
|
||||
|
||||
Using [Augmented Backus-Naur form (ABNF)](https://tools.ietf.org/html/rfc5234) we have the following format:
|
||||
|
||||
```abnf
|
||||
; 1 byte; first two bits contain the size of auxiliary field,
|
||||
; third bit indicates whether the signature is present.
|
||||
flags = 1OCTET
|
||||
|
||||
; contains the size of payload.
|
||||
auxiliary-field = 4*OCTET
|
||||
|
||||
; byte array of arbitrary size (may be zero)
|
||||
payload = *OCTET
|
||||
|
||||
; byte array of arbitrary size (may be zero).
|
||||
padding = *OCTET
|
||||
|
||||
; 65 bytes, if present.
|
||||
signature = 65OCTET
|
||||
|
||||
; 2 bytes, if present (in case of symmetric encryption).
|
||||
salt = 2OCTET
|
||||
|
||||
data = flags auxiliary-field payload padding [signature] [salt]
|
||||
```
|
||||
|
||||
### Signature
|
||||
|
||||
Those unable to decrypt the envelope data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialized as the concatenation of the `R`, `S` and `V` parameters of the SECP-256k1 ECDSA signature, in that order. `R` and `S` MUST be big-endian encoded, fixed-width 256-bit unsigned. `V` MUST be an 8-bit big-endian encoded, non-normalized and should be either 27 or 28.
|
||||
|
||||
### Padding
|
||||
|
||||
The padding field is used to align data size, since data size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the Salt) before encryption (i.e. plain text) SHOULD be factor of 256 bytes.
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,118 @@
|
|||
---
|
||||
slug: 8
|
||||
title: 8/WAKU-MAIL
|
||||
name: Waku Mailserver
|
||||
status: stable
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Adam Babik <adam@status.im>
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
---
|
||||
|
||||
## Abstract
|
||||
|
||||
In this specification, we describe Mailservers. These are nodes responsible for archiving envelopes and delivering them to peers on-demand.
|
||||
|
||||
## Specification
|
||||
|
||||
A node which wants to provide mailserver functionality MUST store envelopes from incoming Messages packets (Waku packet-code `0x01`). The envelopes can be stored in any format, however they MUST be serialized and deserialized to the Waku envelope format.
|
||||
|
||||
A mailserver SHOULD store envelopes for all topics to be generally useful for any peer, however for specific use cases it MAY store envelopes for a subset of topics.
|
||||
|
||||
### Requesting Historic Envelopes
|
||||
|
||||
In order to request historic envelopes, a node MUST send a packet P2P Request (`0x7e`) to a peer providing mailserver functionality. This packet requires one argument which MUST be a Waku envelope.
|
||||
|
||||
In the Waku envelope's payload section, there MUST be RLP-encoded information about the details of the request:
|
||||
|
||||
```abnf
|
||||
; UNIX time in seconds; oldest requested envelope's creation time
|
||||
lower = 4OCTET
|
||||
|
||||
; UNIX time in seconds; newest requested envelope's creation time
|
||||
upper = 4OCTET
|
||||
|
||||
; array of Waku topics encoded in a bloom filter to filter envelopes
|
||||
bloom = 64OCTET
|
||||
|
||||
; unsigned integer limiting the number of returned envelopes
|
||||
limit = 4OCTET
|
||||
|
||||
; array of a cursor returned from the previous request (optional)
|
||||
cursor = *OCTET
|
||||
|
||||
; List of topics interested in
|
||||
topics = "[" *1000topic "]"
|
||||
|
||||
; 4 bytes of arbitrary data
|
||||
topic = 4OCTET
|
||||
|
||||
payload-without-topic = "[" lower upper bloom limit [ cursor ] "]"
|
||||
|
||||
payload-with-topic = "[" lower upper bloom limit cursor [ topics ] "]"
|
||||
|
||||
payload = payload-with-topic | payload-without-topic
|
||||
```
|
||||
|
||||
The `Cursor` field SHOULD be filled in if a number of envelopes between `Lower` and `Upper` is greater than `Limit` so that the requester can send another request using the obtained `Cursor` value. What exactly is in the `Cursor` is up to the implementation. The requester SHOULD NOT use a `Cursor` obtained from one mailserver in a request to another mailserver because the format or the result MAY be different.
|
||||
|
||||
The envelope MUST be encrypted with a symmetric key agreed between the requester and Mailserver.
|
||||
|
||||
If `Topics` is used the `Cursor` field MUST be specified for the argument order to be unambiguous. However, it MAY be set to `null`. `Topics` is used to specify which topics a node is interested in. If `Topics` is not empty, a mailserver MUST only send envelopes that belong to a topic from `Topics` list and `Bloom` value MUST be ignored.
|
||||
|
||||
### Receiving Historic Envelopes
|
||||
|
||||
Historic envelopes MUST be sent to a peer as a packet with a P2P Message code (`0x7f`) followed by an array of Waku envelopes. A Mailserver MUST limit the amount of messages sent, either by the `Limit` specified in the request or limited to the maximum [RLPx packet size](./waku#maximum-packet-size), whichever limit comes first.
|
||||
|
||||
In order to receive historic envelopes from a mailserver, a node MUST trust the selected mailserver, that is allow to receive expired packets with the P2P Message code. By default, such packets are discarded.
|
||||
|
||||
Received envelopes MUST be passed through the Whisper envelope pipelines so that they are picked up by registered filters and passed to subscribers.
|
||||
|
||||
For a requester, to know that all envelopes have been sent by mailserver, it SHOULD handle P2P Request Complete code (`0x7d`). This code is followed by a list with:
|
||||
|
||||
```abnf
|
||||
; array with a Keccak-256 hash of the envelope containing the original request.
|
||||
request-id = 32OCTET
|
||||
|
||||
; array with a Keccak-256 hash of the last sent envelope for the request.
|
||||
last-envelope-hash = 32OCTET
|
||||
|
||||
; array of a cursor returned from the previous request (optional)
|
||||
cursor = *OCTET
|
||||
|
||||
payload = "[" request-id last-envelope-hash [ cursor ] "]"
|
||||
```
|
||||
|
||||
If `Cursor` is not empty, it means that not all envelopes were sent due to the set `Limit` in the request. One or more consecutive requests MAY be sent with `Cursor` field filled in in order to receive the rest of the envelopes.
|
||||
|
||||
### Security considerations
|
||||
|
||||
There are several security considerations to take into account when running or interacting with Mailservers. Chief among them are: scalability, DDoS-resistance and privacy.
|
||||
|
||||
**Mailserver High Availability requirement:**
|
||||
|
||||
A mailserver has to be online to receive envelopes for other nodes, this puts a high availability requirement on it.
|
||||
|
||||
**Mailserver client privacy:**
|
||||
|
||||
A mailserver client has to trust a mailserver, which means they can send direct traffic. This reveals what topics / bloom filter a node is interested in, along with its peerID (with IP).
|
||||
|
||||
**Mailserver trusted connection:**
|
||||
|
||||
A mailserver has a direct TCP connection, which means they are trusted to send traffic. This means a malicious or malfunctioning mailserver can overwhelm an individual node.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Version | Comment |
|
||||
| :--------------------------------------------------------------------------------------------: | ------- |
|
||||
| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8) | marked stable as it is in use. |
|
||||
| 0.2.0 | Add topic interest to reduce bandwidth usage |
|
||||
| [0.1.0](https://github.com/vacp2p/specs/blob/06d4c736c920526472a507e5d842212843a112ed/wms.md) | Initial Release |
|
||||
|
||||
### Difference between wms 0.1 and wms 0.2
|
||||
|
||||
- `topics` option
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,408 @@
|
|||
---
|
||||
slug: 9
|
||||
title: 9/WAKU-RPC
|
||||
name: Waku RPC API
|
||||
status: stable
|
||||
editor: Oskar Thorén <oskar@status.im>
|
||||
contributors:
|
||||
- Dean Eigenmann <dean@status.im>
|
||||
---
|
||||
|
||||
This specification describes the RPC API that Waku nodes MAY adhere to. The unified API allows clients to easily
|
||||
be able to connect to any node implementation. The API described is privileged as a node stores the keys of clients.
|
||||
|
||||
## Introduction
|
||||
|
||||
This API is based off the [Whisper V6 RPC API](https://github.com/ethereum/go-ethereum/wiki/Whisper-v6-RPC-API).
|
||||
|
||||
## Wire Protocol
|
||||
|
||||
### Transport
|
||||
|
||||
Nodes SHOULD expose a [JSON RPC](https://www.jsonrpc.org/specification) API that can be accessed. The JSON RPC version SHOULD be `2.0`. Below is an example request:
|
||||
|
||||
```json
|
||||
{
|
||||
"jsonrpc":"2.0",
|
||||
"method":"waku_version",
|
||||
"params":[],
|
||||
"id":1
|
||||
}
|
||||
```
|
||||
|
||||
#### Fields
|
||||
|
||||
| Field | Description |
|
||||
| --------- | --------------------------------------------------- |
|
||||
| `jsonrpc` | Contains the used JSON RPC version (`Default: 2.0`) |
|
||||
| `method` | Contains the JSON RPC method that is being called |
|
||||
| `params` | An array of parameters for the request |
|
||||
| `id` | The request ID |
|
||||
|
||||
### Objects
|
||||
|
||||
In this section you will find objects used throughout the JSON RPC API.
|
||||
|
||||
#### Message
|
||||
|
||||
The message object represents a Waku message. Below you will find the description of the attributes contained in the message object. A message is the decrypted payload and padding of an [envelope](/spec/7) along with all of its metadata and other extra information such as the hash.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ----: | :--: | ----------- |
|
||||
| `sig` | string | Public Key that signed this message |
|
||||
| `recipientPublicKey` | string | The recipients public key |
|
||||
| `ttl` | number | Time-to-live in seconds |
|
||||
| `timestamp` | number | Unix timestamp of the message generation |
|
||||
| `topic` | string | 4 bytes, the message topic |
|
||||
| `payload` | string | Decrypted payload |
|
||||
| `padding` | string | Optional padding, byte array of arbitrary length |
|
||||
| `pow` | number | The proof of work value |
|
||||
| `hash` | string | Hash of the enveloped message |
|
||||
|
||||
#### Filter
|
||||
|
||||
The filter object represents filters that can be applied to retrieve messages. Below you will find the description of the attributes contained in the filter object.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ----: | :--: | ----------- |
|
||||
| `symKeyID` | string | ID of the symmetric key for message decryption |
|
||||
| `privateKeyID` | string | ID of private (asymmetric) key for message decryption |
|
||||
| `sig` | string | Public key of the signature |
|
||||
| `minPow` | number | Minimal PoW requirement for incoming messages |
|
||||
| `topics` | array | Array of possible topics, this can also contain partial topics |
|
||||
| `allowP2P` | boolean | Indicates if this filter allows processing of direct peer-to-peer messages |
|
||||
|
||||
All fields are optional, however `symKeyID` or `privateKeyID` must be present, it cannot be both. Additionally, the `topics` field is only optional when an asymmetric key is used.
|
||||
|
||||
### Methods
|
||||
|
||||
#### `waku_version`
|
||||
|
||||
The `waku_version` method returns the current version number.
|
||||
|
||||
##### Parameters
|
||||
|
||||
none
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The version number.
|
||||
|
||||
#### `waku_info`
|
||||
|
||||
The `waku_info` method returns information about a Waku node.
|
||||
|
||||
##### Parameters
|
||||
|
||||
none
|
||||
|
||||
##### Response
|
||||
|
||||
The response is an `Object` containing the following fields:
|
||||
|
||||
- **`minPow` [number]** - The current PoW requirement.
|
||||
- **`maxEnvelopeSize` [float]** - The current maximum envelope size in bytes.
|
||||
- **`memory` [number]** - The memory size of the floating messages in bytes.
|
||||
- **`envelopes` [number]** - The number of floating envelopes.
|
||||
|
||||
#### `waku_setMaxEnvelopeSize`
|
||||
|
||||
Sets the maximum envelope size allowed by this node. Any envelopes larger than this size both incoming and outgoing will be rejected. The envelope size can never exceed the underlying envelope size of `10mb`.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **number** - The message size in bytes.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_setMinPoW`
|
||||
|
||||
Sets the minimal PoW required by this node.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **number** - The new PoW requirement.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_markTrustedPeer`
|
||||
|
||||
Marks a specific peer as trusted allowing it to send expired messages.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - `enode` of the peer.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_newKeyPair`
|
||||
|
||||
Generates a keypair used for message encryption and decryption.
|
||||
|
||||
##### Parameters
|
||||
|
||||
none
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_addPrivateKey`
|
||||
|
||||
Stores a key and returns its ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - Private key as hex bytes.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - Key ID on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_deleteKeyPair`
|
||||
|
||||
Deletes a specific key if it exists.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key pair.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_hasKeyPair`
|
||||
|
||||
Checks if the node has a private key of a key pair matching the given ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key pair.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_getPublicKey`
|
||||
|
||||
Returns the public key for an ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The public key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_getPrivateKey`
|
||||
|
||||
Returns the private key for an ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The private key or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_newSymKey`
|
||||
|
||||
Generates a random symmetric key and stores it under an ID. This key can be used to encrypt and decrypt messages where the key is known to both parties.
|
||||
|
||||
##### Parameters
|
||||
|
||||
none
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_addSymKey`
|
||||
|
||||
Stores the key and returns its ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - The raw key for symmetric encryption hex encoded.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_generateSymKeyFromPassword`
|
||||
|
||||
Generates the key from a password and stores it.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - The password.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The key ID or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_hasSymKey`
|
||||
|
||||
Returns whether there is a key associated with the ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_getSymKey`
|
||||
|
||||
Returns the symmetric key associated with an ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - Raw key on success or an [error](https://www.jsonrpc.org/specification#error_object) of failure.
|
||||
|
||||
#### `waku_deleteSymKey`
|
||||
|
||||
Deletes the key associated with an ID.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the Key.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` or `false` or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_subscribe`
|
||||
|
||||
Creates and registers a new subscription to receive notifications for inbound Waku messages.
|
||||
|
||||
##### Parameters
|
||||
|
||||
The parameters for this request is an array containing the following fields:
|
||||
|
||||
1. **string** - The ID of the function call, in case of Waku this must contain the value "messages".
|
||||
2. **object** - The [message filter](#filter).
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - ID of the subscription or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
###### Notifications
|
||||
|
||||
Notifications received by the client contain a [message](#message) matching the filter. Below is an example notification:
|
||||
|
||||
```json
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"method": "waku_subscription",
|
||||
"params": {
|
||||
"subscription": "02c1f5c953804acee3b68eda6c0afe3f1b4e0bec73c7445e10d45da333616412",
|
||||
"result": {
|
||||
"sig": "0x0498ac1951b9078a0549c93c3f6088ec7c790032b17580dc3c0c9e900899a48d89eaa27471e3071d2de6a1f48716ecad8b88ee022f4321a7c29b6ffcbee65624ff",
|
||||
"recipientPublicKey": null,
|
||||
"ttl": 10,
|
||||
"timestamp": 1498577270,
|
||||
"topic": "0xffaadd11",
|
||||
"payload": "0xffffffdddddd1122",
|
||||
"padding": "0x35d017b66b124dd4c67623ed0e3f23ba68e3428aa500f77aceb0dbf4b63f69ccfc7ae11e39671d7c94f1ed170193aa3e327158dffdd7abb888b3a3cc48f718773dc0a9dcf1a3680d00fe17ecd4e8d5db31eb9a3c8e6e329d181ecb6ab29eb7a2d9889b49201d9923e6fd99f03807b730780a58924870f541a8a97c87533b1362646e5f573dc48382ef1e70fa19788613c9d2334df3b613f6e024cd7aadc67f681fda6b7a84efdea40cb907371cd3735f9326d02854",
|
||||
"pow": 0.6714754098360656,
|
||||
"hash": "0x17f8066f0540210cf67ef400a8a55bcb32a494a47f91a0d26611c5c1d66f8c57"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `waku_unsubscribe`
|
||||
|
||||
Cancels and removes an existing subscription. The node MUST stop sending the client notifications.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - The subscription ID.
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` or `false`
|
||||
|
||||
#### `waku_newMessageFilter`
|
||||
|
||||
Creates a new message filter within the node. This filter can be used to poll for new messages that match the criteria.
|
||||
|
||||
##### Parameters
|
||||
|
||||
The request must contain a [message filter](#filter) as its parameter.
|
||||
|
||||
##### Response
|
||||
|
||||
- **string** - The ID of the filter.
|
||||
|
||||
#### `waku_deleteMessageFilter`
|
||||
|
||||
Removes a message filter from the node.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
|
||||
|
||||
##### Response
|
||||
|
||||
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_getFilterMessages`
|
||||
|
||||
Retrieves messages that match a filter criteria and were received after the last time this function was called.
|
||||
|
||||
##### Parameters
|
||||
|
||||
- **string** - ID of the filter created with [`waku_newMessageFilter`](#waku_newMessageFilter).
|
||||
|
||||
##### Response
|
||||
|
||||
The response contains an array of [messages](#messages) or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
#### `waku_post`
|
||||
|
||||
The `waku_post` method creates a waku envelope and propagates it to the network.
|
||||
|
||||
##### Parameters
|
||||
|
||||
The parameters is an `Object` containing the following fields:
|
||||
- **`symKeyID` [string]** `optional` - The ID of the symmetric key used for encryption
|
||||
- **`pubKey` [string]** `optional` - The public key for message encryption.
|
||||
- **`sig` [string]** `optional` - The ID of the signing key.
|
||||
- **`ttl` [number]** - The time-to-live in seconds.
|
||||
- **`topic` [string]** - 4 bytes message topic.
|
||||
- **`payload` [string]** - The payload to be encrypted.
|
||||
- **`padding` [string]** `optional` - The padding, a byte array of arbitrary length.
|
||||
- **`powTime` [number]** - Maximum time in seconds to be spent on the proof of work.
|
||||
- **`powTarget` [number]** - Minimal PoW target required for this message.
|
||||
- **`targetPeer` [string]** `optional` - The optional peer ID for peer-to-peer messages.
|
||||
|
||||
*Either the **`symKeyID`** or the **`pubKey`** need to be present. It can not be both.*
|
||||
|
||||
#### Response
|
||||
|
||||
- **bool** - `true` on success or an [error](https://www.jsonrpc.org/specification#error_object) on failure.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Version | Comment |
|
||||
| :--------------------------------------------------------------------------------------:| ---------------- |
|
||||
| [1.0.0](https://github.com/vacp2p/specs/commit/bc7e75ebb2e45d2cbf6ab27352c113e666df37c8)| Initial release. |
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
|
@ -0,0 +1,28 @@
|
|||
---
|
||||
headless: true
|
||||
bookMenuLevels: 1
|
||||
---
|
||||
|
||||
- Raw
|
||||
- [15/WAKU2-BRIDGE]({{< relref "/docs/rfcs/15/README.md" >}})
|
||||
- [16/WAKU2-RPC]({{< relref "/docs/rfcs/16/README.md" >}})
|
||||
- [17/WAKU2-RLNRELAY]({{< relref "/docs/rfcs/17/README.md" >}})
|
||||
- [18/WAKU2-SWAP]({{< relref "/docs/rfcs/18/README.md" >}})
|
||||
- Draft
|
||||
- [1/COSS]({{< relref "/docs/rfcs/1/README.md" >}})
|
||||
- [3/REMOTE-LOG]({{< relref "/docs/rfcs/3/README.md" >}})
|
||||
- [4/MVDS-META]({{< relref "/docs/rfcs/4/README.md" >}})
|
||||
- [10/WAKU2]({{< relref "/docs/rfcs/10/README.md" >}})
|
||||
- [11/WAKU2-RELAY]({{< relref "/docs/rfcs/11/README.md" >}})
|
||||
- [12/WAKU2-FILTER]({{< relref "/docs/rfcs/12/README.md" >}})
|
||||
- [13/WAKU2-STORE]({{< relref "/docs/rfcs/13/README.md" >}})
|
||||
- [14/WAKU2-MESSAGE]({{< relref "/docs/rfcs/14/README.md" >}})
|
||||
- Stable
|
||||
- [2/MVDS]({{< relref "/docs/rfcs/2/README.md" >}})
|
||||
- [6/WAKU1]({{< relref "/docs/rfcs/6/README.md" >}})
|
||||
- [7/WAKU-DATA]({{< relref "/docs/rfcs/7/README.md" >}})
|
||||
- [8/WAKU-MAIL]({{< relref "/docs/rfcs/8/README.md" >}})
|
||||
- [9/WAKU-RPC]({{< relref "/docs/rfcs/9/README.md" >}})
|
||||
- Deprecated
|
||||
- [5/WAKU0]({{< relref "/docs/rfcs/5/README.md" >}})
|
||||
- Retired
|
|
@ -0,0 +1,3 @@
|
|||
# About
|
||||
|
||||
Example content from template site. Kept around to have examples of formatting, pages etc. Separate directory to not mess with live content.
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
title: Introduction
|
||||
type: docs
|
||||
---
|
||||
|
||||
# Acerbo datus maxime
|
||||
|
||||
{{< columns >}}
|
||||
## Astris ipse furtiva
|
||||
|
||||
Est in vagis et Pittheus tu arge accipiter regia iram vocatur nurus. Omnes ut
|
||||
olivae sensit **arma sorori** deducit, inesset **crudus**, ego vetuere aliis,
|
||||
modo arsit? Utinam rapta fiducia valuere litora _adicit cursu_, ad facies
|
||||
|
||||
<--->
|
||||
|
||||
## Suis quot vota
|
||||
|
||||
Ea _furtique_ risere fratres edidit terrae magis. Colla tam mihi tenebat:
|
||||
miseram excita suadent es pecudes iam. Concilio _quam_ velatus posset ait quod
|
||||
nunc! Fragosis suae dextra geruntur functus vulgata.
|
||||
{{< /columns >}}
|
||||
|
||||
|
||||
## Tempora nisi nunc
|
||||
|
||||
Lorem **markdownum** emicat gestu. Cannis sol pressit ducta. **Est** Idaei,
|
||||
tremens ausim se tutaeque, illi ulnis hausit, sed, lumina cutem. Quae avis
|
||||
sequens!
|
||||
|
||||
var panel = ram_design;
|
||||
if (backup + system) {
|
||||
file.readPoint = network_native;
|
||||
sidebar_engine_device(cell_tftp_raster,
|
||||
dual_login_paper.adf_vci.application_reader_design(
|
||||
graphicsNvramCdma, lpi_footer_snmp, integer_model));
|
||||
}
|
||||
|
||||
## Locis suis novi cum suoque decidit eadem
|
||||
|
||||
Idmoniae ripis, at aves, ali missa adest, ut _et autem_, et ab?
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
weight: 1
|
||||
bookFlatSection: true
|
||||
title: "Example Site"
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
## Ferre hinnitibus erat accipitrem dixi Troiae tollens
|
||||
|
||||
Lorem markdownum, a quoque nutu est *quodcumque mandasset* veluti. Passim
|
||||
inportuna totidemque nympha fert; repetens pendent, poenarum guttura sed vacet
|
||||
non, mortali undas. Omnis pharetramque gramen portentificisque membris servatum
|
||||
novabis fallit de nubibus atque silvas mihi. **Dixit repetitaque Quid**; verrit
|
||||
longa; sententia [mandat](http://pastor-ad.io/questussilvas) quascumque nescio
|
||||
solebat [litore](http://lacrimas-ab.net/); noctes. *Hostem haerentem* circuit
|
||||
[plenaque tamen](http://www.sine.io/in).
|
||||
|
||||
- Pedum ne indigenae finire invergens carpebat
|
||||
- Velit posses summoque
|
||||
- De fumos illa foret
|
||||
|
||||
## Est simul fameque tauri qua ad
|
||||
|
||||
Locum nullus nisi vomentes. Ab Persea sermone vela, miratur aratro; eandem
|
||||
Argolicas gener.
|
||||
|
||||
## Me sol
|
||||
|
||||
Nec dis certa fuit socer, Nonacria **dies** manet tacitaque sibi? Sucis est
|
||||
iactata Castrumque iudex, et iactato quoque terraeque es tandem et maternos
|
||||
vittis. Lumina litus bene poenamque animos callem ne tuas in leones illam dea
|
||||
cadunt genus, et pleno nunc in quod. Anumque crescentesque sanguinis
|
||||
[progenies](http://www.late.net/alimentavirides) nuribus rustica tinguet. Pater
|
||||
omnes liquido creditis noctem.
|
||||
|
||||
if (mirrored(icmp_dvd_pim, 3, smbMirroredHard) != lion(clickImportQueue,
|
||||
viralItunesBalancing, bankruptcy_file_pptp)) {
|
||||
file += ip_cybercrime_suffix;
|
||||
}
|
||||
if (runtimeSmartRom == netMarketingWord) {
|
||||
virusBalancingWin *= scriptPromptBespoke + raster(post_drive,
|
||||
windowsSli);
|
||||
cd = address_hertz_trojan;
|
||||
soap_ccd.pcbServerGigahertz(asp_hardware_isa, offlinePeopleware, nui);
|
||||
} else {
|
||||
megabyte.api = modem_flowchart - web + syntaxHalftoneAddress;
|
||||
}
|
||||
if (3 < mebibyteNetworkAnimated) {
|
||||
pharming_regular_error *= jsp_ribbon + algorithm * recycleMediaKindle(
|
||||
dvrSyntax, cdma);
|
||||
adf_sla *= hoverCropDrive;
|
||||
templateNtfs = -1 - vertical;
|
||||
} else {
|
||||
expressionCompressionVariable.bootMulti = white_eup_javascript(
|
||||
table_suffix);
|
||||
guidPpiPram.tracerouteLinux += rtfTerabyteQuicktime(1,
|
||||
managementRosetta(webcamActivex), 740874);
|
||||
}
|
||||
var virusTweetSsl = nullGigo;
|
||||
|
||||
## Trepident sitimque
|
||||
|
||||
Sentiet et ferali errorem fessam, coercet superbus, Ascaniumque in pennis
|
||||
mediis; dolor? Vidit imi **Aeacon** perfida propositos adde, tua Somni Fluctibus
|
||||
errante lustrat non.
|
||||
|
||||
Tamen inde, vos videt e flammis Scythica parantem rupisque pectora umbras. Haec
|
||||
ficta canistris repercusso simul ego aris Dixit! Esse Fama trepidare hunc
|
||||
crescendo vigor ululasse vertice *exspatiantur* celer tepidique petita aversata
|
||||
oculis iussa est me ferro.
|
|
@ -0,0 +1,12 @@
|
|||
# 4th Level of Menu
|
||||
|
||||
## Caesorum illa tu sentit micat vestes papyriferi
|
||||
|
||||
Inde aderam facti; Theseus vis de tauri illa peream. Oculos **uberaque** non
|
||||
regisque vobis cursuque, opus venit quam vulnera. Et maiora necemque, lege modo;
|
||||
gestanda nitidi, vero? Dum ne pectoraque testantur.
|
||||
|
||||
Venasque repulsa Samos qui, exspectatum eram animosque hinc, [aut
|
||||
manes](http://www.creveratnon.net/apricaaetheriis), Assyrii. Cupiens auctoribus
|
||||
pariter rubet, profana magni super nocens. Vos ius sibilat inpar turba visae
|
||||
iusto! Sedes ante dum superest **extrema**.
|
|
@ -0,0 +1,26 @@
|
|||
# 3rd Level of Menu
|
||||
|
||||
Nefas discordemque domino montes numen tum humili nexilibusque exit, Iove. Quae
|
||||
miror esse, scelerisque Melaneus viribus. Miseri laurus. Hoc est proposita me
|
||||
ante aliquid, aura inponere candidioribus quidque accendit bella, sumpta.
|
||||
Intravit quam erat figentem hunc, motus de fontes parvo tempestate.
|
||||
|
||||
iscsi_virus = pitch(json_in_on(eupViral),
|
||||
northbridge_services_troubleshooting, personal(
|
||||
firmware_rw.trash_rw_crm.device(interactive_gopher_personal,
|
||||
software, -1), megabit, ergonomicsSoftware(cmyk_usb_panel,
|
||||
mips_whitelist_duplex, cpa)));
|
||||
if (5) {
|
||||
managementNetwork += dma - boolean;
|
||||
kilohertz_token = 2;
|
||||
honeypot_affiliate_ergonomics = fiber;
|
||||
}
|
||||
mouseNorthbridge = byte(nybble_xmp_modem.horse_subnet(
|
||||
analogThroughputService * graphicPoint, drop(daw_bit, dnsIntranet),
|
||||
gateway_ospf), repository.domain_key.mouse(serverData(fileNetwork,
|
||||
trim_duplex_file), cellTapeDirect, token_tooltip_mashup(
|
||||
ripcordingMashup)));
|
||||
module_it = honeypot_driver(client_cold_dvr(593902, ripping_frequency) +
|
||||
coreLog.joystick(componentUdpLink), windows_expansion_touchscreen);
|
||||
bashGigabit.external.reality(2, server_hardware_codec.flops.ebookSampling(
|
||||
ciscNavigationBacklink, table + cleanDriver), indexProtocolIsp);
|
|
@ -0,0 +1,4 @@
|
|||
---
|
||||
bookCollapseSection: true
|
||||
weight: 20
|
||||
---
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
bookHidden: true
|
||||
---
|
||||
|
||||
# This page is hidden in menu
|
||||
|
||||
# Quondam non pater est dignior ille Eurotas
|
||||
|
||||
## Latent te facies
|
||||
|
||||
Lorem markdownum arma ignoscas vocavit quoque ille texit mandata mentis ultimus,
|
||||
frementes, qui in vel. Hippotades Peleus [pennas
|
||||
conscia](http://gratia.net/tot-qua.php) cuiquam Caeneus quas.
|
||||
|
||||
- Pater demittere evincitque reddunt
|
||||
- Maxime adhuc pressit huc Danaas quid freta
|
||||
- Soror ego
|
||||
- Luctus linguam saxa ultroque prior Tatiumque inquit
|
||||
- Saepe liquitur subita superata dederat Anius sudor
|
||||
|
||||
## Cum honorum Latona
|
||||
|
||||
O fallor [in sustinui
|
||||
iussorum](http://www.spectataharundine.org/aquas-relinquit.html) equidem.
|
||||
Nymphae operi oris alii fronde parens dumque, in auro ait mox ingenti proxima
|
||||
iamdudum maius?
|
||||
|
||||
reality(burnDocking(apache_nanometer),
|
||||
pad.property_data_programming.sectorBrowserPpga(dataMask, 37,
|
||||
recycleRup));
|
||||
intellectualVaporwareUser += -5 * 4;
|
||||
traceroute_key_upnp /= lag_optical(android.smb(thyristorTftp));
|
||||
surge_host_golden = mca_compact_device(dual_dpi_opengl, 33,
|
||||
commerce_add_ppc);
|
||||
if (lun_ipv) {
|
||||
verticalExtranet(1, thumbnail_ttl, 3);
|
||||
bar_graphics_jpeg(chipset - sector_xmp_beta);
|
||||
}
|
||||
|
||||
## Fronde cetera dextrae sequens pennis voce muneris
|
||||
|
||||
Acta cretus diem restet utque; move integer, oscula non inspirat, noctisque
|
||||
scelus! Nantemque in suas vobis quamvis, et labori!
|
||||
|
||||
var runtimeDiskCompiler = home - array_ad_software;
|
||||
if (internic > disk) {
|
||||
emoticonLockCron += 37 + bps - 4;
|
||||
wan_ansi_honeypot.cardGigaflops = artificialStorageCgi;
|
||||
simplex -= downloadAccess;
|
||||
}
|
||||
var volumeHardeningAndroid = pixel + tftp + onProcessorUnmount;
|
||||
sector(memory(firewire + interlaced, wired));
|
|
@ -0,0 +1,85 @@
|
|||
---
|
||||
weight: 10
|
||||
---
|
||||
|
||||
# Ubi loqui
|
||||
|
||||
## Mentem genus facietque salire tempus bracchia
|
||||
|
||||
Lorem markdownum partu paterno Achillem. Habent amne generosi aderant ad pellem
|
||||
nec erat sustinet merces columque haec et, dixit minus nutrit accipiam subibis
|
||||
subdidit. Temeraria servatum agros qui sed fulva facta. Primum ultima, dedit,
|
||||
suo quisque linguae medentes fixo: tum petis.
|
||||
|
||||
## Rapit vocant si hunc siste adspice
|
||||
|
||||
Ora precari Patraeque Neptunia, dixit Danae [Cithaeron
|
||||
armaque](http://mersis-an.org/litoristum) maxima in **nati Coniugis** templis
|
||||
fluidove. Effugit usus nec ingreditur agmen *ac manus* conlato. Nullis vagis
|
||||
nequiquam vultibus aliquos altera *suum venis* teneas fretum. Armos [remotis
|
||||
hoc](http://tutum.io/me) sine ferrea iuncta quam!
|
||||
|
||||
## Locus fuit caecis
|
||||
|
||||
Nefas discordemque domino montes numen tum humili nexilibusque exit, Iove. Quae
|
||||
miror esse, scelerisque Melaneus viribus. Miseri laurus. Hoc est proposita me
|
||||
ante aliquid, aura inponere candidioribus quidque accendit bella, sumpta.
|
||||
Intravit quam erat figentem hunc, motus de fontes parvo tempestate.
|
||||
|
||||
iscsi_virus = pitch(json_in_on(eupViral),
|
||||
northbridge_services_troubleshooting, personal(
|
||||
firmware_rw.trash_rw_crm.device(interactive_gopher_personal,
|
||||
software, -1), megabit, ergonomicsSoftware(cmyk_usb_panel,
|
||||
mips_whitelist_duplex, cpa)));
|
||||
if (5) {
|
||||
managementNetwork += dma - boolean;
|
||||
kilohertz_token = 2;
|
||||
honeypot_affiliate_ergonomics = fiber;
|
||||
}
|
||||
mouseNorthbridge = byte(nybble_xmp_modem.horse_subnet(
|
||||
analogThroughputService * graphicPoint, drop(daw_bit, dnsIntranet),
|
||||
gateway_ospf), repository.domain_key.mouse(serverData(fileNetwork,
|
||||
trim_duplex_file), cellTapeDirect, token_tooltip_mashup(
|
||||
ripcordingMashup)));
|
||||
module_it = honeypot_driver(client_cold_dvr(593902, ripping_frequency) +
|
||||
coreLog.joystick(componentUdpLink), windows_expansion_touchscreen);
|
||||
bashGigabit.external.reality(2, server_hardware_codec.flops.ebookSampling(
|
||||
ciscNavigationBacklink, table + cleanDriver), indexProtocolIsp);
|
||||
|
||||
## Placabilis coactis nega ingemuit ignoscat nimia non
|
||||
|
||||
Frontis turba. Oculi gravis est Delphice; *inque praedaque* sanguine manu non.
|
||||
|
||||
if (ad_api) {
|
||||
zif += usb.tiffAvatarRate(subnet, digital_rt) + exploitDrive;
|
||||
gigaflops(2 - bluetooth, edi_asp_memory.gopher(queryCursor, laptop),
|
||||
panel_point_firmware);
|
||||
spyware_bash.statePopApplet = express_netbios_digital(
|
||||
insertion_troubleshooting.brouter(recordFolderUs), 65);
|
||||
}
|
||||
recursionCoreRay = -5;
|
||||
if (hub == non) {
|
||||
portBoxVirus = soundWeb(recursive_card(rwTechnologyLeopard),
|
||||
font_radcab, guidCmsScalable + reciprocalMatrixPim);
|
||||
left.bug = screenshot;
|
||||
} else {
|
||||
tooltipOpacity = raw_process_permalink(webcamFontUser, -1);
|
||||
executable_router += tape;
|
||||
}
|
||||
if (tft) {
|
||||
bandwidthWeb *= social_page;
|
||||
} else {
|
||||
regular += 611883;
|
||||
thumbnail /= system_lag_keyboard;
|
||||
}
|
||||
|
||||
## Caesorum illa tu sentit micat vestes papyriferi
|
||||
|
||||
Inde aderam facti; Theseus vis de tauri illa peream. Oculos **uberaque** non
|
||||
regisque vobis cursuque, opus venit quam vulnera. Et maiora necemque, lege modo;
|
||||
gestanda nitidi, vero? Dum ne pectoraque testantur.
|
||||
|
||||
Venasque repulsa Samos qui, exspectatum eram animosque hinc, [aut
|
||||
manes](http://www.creveratnon.net/apricaaetheriis), Assyrii. Cupiens auctoribus
|
||||
pariter rubet, profana magni super nocens. Vos ius sibilat inpar turba visae
|
||||
iusto! Sedes ante dum superest **extrema**.
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
title: With ToC
|
||||
weight: 1
|
||||
---
|
||||
# Caput vino delphine in tamen vias
|
||||
|
||||
## Cognita laeva illo fracta
|
||||
|
||||
Lorem markdownum pavent auras, surgit nunc cingentibus libet **Laomedonque que**
|
||||
est. Pastor [An](http://est.org/ire.aspx) arbor filia foedat, ne [fugit
|
||||
aliter](http://www.indiciumturbam.org/moramquid.php), per. Helicona illas et
|
||||
callida neptem est *Oresitrophos* caput, dentibus est venit. Tenet reddite
|
||||
[famuli](http://www.antro-et.net/) praesentem fortibus, quaeque vis foret si
|
||||
frondes *gelidos* gravidae circumtulit [inpulit armenta
|
||||
nativum](http://incurvasustulit.io/illi-virtute.html).
|
||||
|
||||
1. Te at cruciabere vides rubentis manebo
|
||||
2. Maturuit in praetemptat ruborem ignara postquam habitasse
|
||||
3. Subitarum supplevit quoque fontesque venabula spretis modo
|
||||
4. Montis tot est mali quasque gravis
|
||||
5. Quinquennem domus arsit ipse
|
||||
6. Pellem turis pugnabant locavit
|
||||
|
||||
## Natus quaerere
|
||||
|
||||
Pectora et sine mulcere, coniuge dum tincta incurvae. Quis iam; est dextra
|
||||
Peneosque, metuis a verba, primo. Illa sed colloque suis: magno: gramen, aera
|
||||
excutiunt concipit.
|
||||
|
||||
> Phrygiae petendo suisque extimuit, super, pars quod audet! Turba negarem.
|
||||
> Fuerat attonitus; et dextra retinet sidera ulnas undas instimulat vacuae
|
||||
> generis? *Agnus* dabat et ignotis dextera, sic tibi pacis **feriente at mora**
|
||||
> euhoeque *comites hostem* vestras Phineus. Vultuque sanguine dominoque [metuit
|
||||
> risi](http://iuvat.org/eundem.php) fama vergit summaque meus clarissimus
|
||||
> artesque tinguebat successor nominis cervice caelicolae.
|
||||
|
||||
## Limitibus misere sit
|
||||
|
||||
Aurea non fata repertis praerupit feruntur simul, meae hosti lentaque *citius
|
||||
levibus*, cum sede dixit, Phaethon texta. *Albentibus summos* multifidasque
|
||||
iungitur loquendi an pectore, mihi ursaque omnia adfata, aeno parvumque in animi
|
||||
perlucentes. Epytus agis ait vixque clamat ornum adversam spondet, quid sceptra
|
||||
ipsum **est**. Reseret nec; saeva suo passu debentia linguam terga et aures et
|
||||
cervix [de](http://www.amnem.io/pervenit.aspx) ubera. Coercet gelidumque manus,
|
||||
doluit volvitur induta?
|
||||
|
||||
## Enim sua
|
||||
|
||||
Iuvenilior filia inlustre templa quidem herbis permittat trahens huic. In
|
||||
cruribus proceres sole crescitque *fata*, quos quos; merui maris se non tamen
|
||||
in, mea.
|
||||
|
||||
## Germana aves pignus tecta
|
||||
|
||||
Mortalia rudibusque caelum cognosceret tantum aquis redito felicior texit, nec,
|
||||
aris parvo acre. Me parum contulerant multi tenentem, gratissime suis; vultum tu
|
||||
occupat deficeret corpora, sonum. E Actaea inplevit Phinea concepit nomenque
|
||||
potest sanguine captam nulla et, in duxisses campis non; mercede. Dicere cur
|
||||
Leucothoen obitum?
|
||||
|
||||
Postibus mittam est *nubibus principium pluma*, exsecratur facta et. Iunge
|
||||
Mnemonidas pallamque pars; vere restitit alis flumina quae **quoque**, est
|
||||
ignara infestus Pyrrha. Di ducis terris maculatum At sede praemia manes
|
||||
nullaque!
|
|
@ -0,0 +1,59 @@
|
|||
---
|
||||
title: Without ToC
|
||||
weight: 2
|
||||
bookToc: false
|
||||
---
|
||||
|
||||
# At me ipso nepotibus nunc celebratior genus
|
||||
|
||||
## Tanto oblite
|
||||
|
||||
Lorem markdownum pectora novis patenti igne sua opus aurae feras materiaque
|
||||
illic demersit imago et aristas questaque posset. Vomit quoque suo inhaesuro
|
||||
clara. Esse cumque, per referri triste. Ut exponit solisque communis in tendens
|
||||
vincetis agisque iamque huic bene ante vetat omina Thebae rates. Aeacus servat
|
||||
admonitu concidit, ad resimas vultus et rugas vultu **dignamque** Siphnon.
|
||||
|
||||
Quam iugulum regia simulacra, plus meruit humo pecorumque haesit, ab discedunt
|
||||
dixit: ritu pharetramque. Exul Laurenti orantem modo, per densum missisque labor
|
||||
manibus non colla unum, obiectat. Tu pervia collo, fessus quae Cretenque Myconon
|
||||
crate! Tegumenque quae invisi sudore per vocari quaque plus ventis fluidos. Nodo
|
||||
perque, fugisse pectora sorores.
|
||||
|
||||
## Summe promissa supple vadit lenius
|
||||
|
||||
Quibus largis latebris aethera versato est, ait sentiat faciemque. Aequata alis
|
||||
nec Caeneus exululat inclite corpus est, ire **tibi** ostendens et tibi. Rigent
|
||||
et vires dique possent lumina; **eadem** dixit poma funeribus paret et felix
|
||||
reddebant ventis utile lignum.
|
||||
|
||||
1. Remansit notam Stygia feroxque
|
||||
2. Et dabit materna
|
||||
3. Vipereas Phrygiaeque umbram sollicito cruore conlucere suus
|
||||
4. Quarum Elis corniger
|
||||
5. Nec ieiunia dixit
|
||||
|
||||
Vertitur mos ortu ramosam contudit dumque; placabat ac lumen. Coniunx Amoris
|
||||
spatium poenamque cavernis Thebae Pleiadasque ponunt, rapiare cum quae parum
|
||||
nimium rima.
|
||||
|
||||
## Quidem resupinus inducto solebat una facinus quae
|
||||
|
||||
Credulitas iniqua praepetibus paruit prospexit, voce poena, sub rupit sinuatur,
|
||||
quin suum ventorumque arcadiae priori. Soporiferam erat formamque, fecit,
|
||||
invergens, nymphae mutat fessas ait finge.
|
||||
|
||||
1. Baculum mandataque ne addere capiti violentior
|
||||
2. Altera duas quam hoc ille tenues inquit
|
||||
3. Sicula sidereus latrantis domoque ratae polluit comites
|
||||
4. Possit oro clausura namque se nunc iuvenisque
|
||||
5. Faciem posuit
|
||||
6. Quodque cum ponunt novercae nata vestrae aratra
|
||||
|
||||
Ite extrema Phrygiis, patre dentibus, tonso perculit, enim blanda, manibus fide
|
||||
quos caput armis, posse! Nocendo fas Alcyonae lacertis structa ferarum manus
|
||||
fulmen dubius, saxa caelum effuge extremis fixum tumor adfecit **bella**,
|
||||
potentes? Dum nec insidiosa tempora tegit
|
||||
[spirarunt](http://mihiferre.net/iuvenes-peto.html). Per lupi pars foliis,
|
||||
porreximus humum negant sunt subposuere Sidone steterant auro. Memoraverit sine:
|
||||
ferrum idem Orion caelum heres gerebat fixis?
|
|
@ -0,0 +1,3 @@
|
|||
---
|
||||
bookFlatSection: true
|
||||
---
|
|
@ -0,0 +1,13 @@
|
|||
# Buttons
|
||||
|
||||
Buttons are styled links that can lead to local page or external link.
|
||||
|
||||
## Example
|
||||
|
||||
```tpl
|
||||
{{</* button relref="/" [class="..."] */>}}Get Home{{</* /button */>}}
|
||||
{{</* button href="https://github.com/alex-shpak/hugo-book" */>}}Contribute{{</* /button */>}}
|
||||
```
|
||||
|
||||
{{< button relref="/" >}}Get Home{{< /button >}}
|
||||
{{< button href="https://github.com/alex-shpak/hugo-book" >}}Contribute{{< /button >}}
|
|
@ -0,0 +1,45 @@
|
|||
# Columns
|
||||
|
||||
Columns help organize shorter pieces of content horizontally for readability.
|
||||
|
||||
|
||||
```html
|
||||
{{</* columns */>}} <!-- begin columns block -->
|
||||
# Left Content
|
||||
Lorem markdownum insigne...
|
||||
|
||||
<---> <!-- magic separator, between columns -->
|
||||
|
||||
# Mid Content
|
||||
Lorem markdownum insigne...
|
||||
|
||||
<---> <!-- magic separator, between columns -->
|
||||
|
||||
# Right Content
|
||||
Lorem markdownum insigne...
|
||||
{{</* /columns */>}}
|
||||
```
|
||||
|
||||
## Example
|
||||
|
||||
{{< columns >}}
|
||||
## Left Content
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
protulit, sed sed aere valvis inhaesuro Pallas animam: qui _quid_, ignes.
|
||||
Miseratus fonte Ditis conubia.
|
||||
|
||||
<--->
|
||||
|
||||
## Mid Content
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter!
|
||||
|
||||
<--->
|
||||
|
||||
## Right Content
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
protulit, sed sed aere valvis inhaesuro Pallas animam: qui _quid_, ignes.
|
||||
Miseratus fonte Ditis conubia.
|
||||
{{< /columns >}}
|
|
@ -0,0 +1,22 @@
|
|||
# Details
|
||||
|
||||
Details shortcode is a helper for `details` html5 element. It is going to replace `expand` shortcode.
|
||||
|
||||
## Example
|
||||
```tpl
|
||||
{{</* details "Title" [open] */>}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne...
|
||||
{{</* /details */>}}
|
||||
```
|
||||
```tpl
|
||||
{{</* details title="Title" open=true */>}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne...
|
||||
{{</* /details */>}}
|
||||
```
|
||||
|
||||
{{< details "Title" open >}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne...
|
||||
{{< /details >}}
|
|
@ -0,0 +1,35 @@
|
|||
# Expand
|
||||
|
||||
Expand shortcode can help to decrease clutter on screen by hiding part of text. Expand content by clicking on it.
|
||||
|
||||
## Example
|
||||
### Default
|
||||
|
||||
```tpl
|
||||
{{</* expand */>}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne...
|
||||
{{</* /expand */>}}
|
||||
```
|
||||
|
||||
{{< expand >}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne...
|
||||
{{< /expand >}}
|
||||
|
||||
### With Custom Label
|
||||
|
||||
```tpl
|
||||
{{</* expand "Custom Label" "..." */>}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne...
|
||||
{{</* /expand */>}}
|
||||
```
|
||||
|
||||
{{< expand "Custom Label" "..." >}}
|
||||
## Markdown content
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
protulit, sed sed aere valvis inhaesuro Pallas animam: qui _quid_, ignes.
|
||||
Miseratus fonte Ditis conubia.
|
||||
{{< /expand >}}
|
|
@ -0,0 +1,32 @@
|
|||
# Hints
|
||||
|
||||
Hint shortcode can be used as hint/alerts/notification block.
|
||||
There are 3 colors to choose: `info`, `warning` and `danger`.
|
||||
|
||||
```tpl
|
||||
{{</* hint [info|warning|danger] */>}}
|
||||
**Markdown content**
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
{{</* /hint */>}}
|
||||
```
|
||||
|
||||
## Example
|
||||
|
||||
{{< hint info >}}
|
||||
**Markdown content**
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
{{< /hint >}}
|
||||
|
||||
{{< hint warning >}}
|
||||
**Markdown content**
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
{{< /hint >}}
|
||||
|
||||
{{< hint danger >}}
|
||||
**Markdown content**
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
{{< /hint >}}
|
|
@ -0,0 +1,28 @@
|
|||
# KaTeX
|
||||
|
||||
KaTeX shortcode let you render math typesetting in markdown document. See [KaTeX](https://katex.org/)
|
||||
|
||||
## Example
|
||||
{{< columns >}}
|
||||
|
||||
```latex
|
||||
{{</* katex [display] [class="text-center"] */>}}
|
||||
f(x) = \int_{-\infty}^\infty\hat f(\xi)\,e^{2 \pi i \xi x}\,d\xi
|
||||
{{</* /katex */>}}
|
||||
```
|
||||
|
||||
<--->
|
||||
|
||||
{{< katex display >}}
|
||||
f(x) = \int_{-\infty}^\infty\hat f(\xi)\,e^{2 \pi i \xi x}\,d\xi
|
||||
{{< /katex >}}
|
||||
|
||||
{{< /columns >}}
|
||||
|
||||
## Display Mode Example
|
||||
|
||||
Here is some inline example: {{< katex >}}\pi(x){{< /katex >}}, rendered in the same line. And below is `display` example, having `display: block`
|
||||
{{< katex display >}}
|
||||
f(x) = \int_{-\infty}^\infty\hat f(\xi)\,e^{2 \pi i \xi x}\,d\xi
|
||||
{{< /katex >}}
|
||||
Text continues here.
|
|
@ -0,0 +1,38 @@
|
|||
# Mermaid Chart
|
||||
|
||||
[Mermaid](https://mermaidjs.github.io/) is library for generating svg charts and diagrams from text.
|
||||
|
||||
## Example
|
||||
|
||||
{{< columns >}}
|
||||
```tpl
|
||||
{{</* mermaid [class="text-center"]*/>}}
|
||||
sequenceDiagram
|
||||
Alice->>Bob: Hello Bob, how are you?
|
||||
alt is sick
|
||||
Bob->>Alice: Not so good :(
|
||||
else is well
|
||||
Bob->>Alice: Feeling fresh like a daisy
|
||||
end
|
||||
opt Extra response
|
||||
Bob->>Alice: Thanks for asking
|
||||
end
|
||||
{{</* /mermaid */>}}
|
||||
```
|
||||
|
||||
<--->
|
||||
|
||||
{{< mermaid >}}
|
||||
sequenceDiagram
|
||||
Alice->>Bob: Hello Bob, how are you?
|
||||
alt is sick
|
||||
Bob->>Alice: Not so good :(
|
||||
else is well
|
||||
Bob->>Alice: Feeling fresh like a daisy
|
||||
end
|
||||
opt Extra response
|
||||
Bob->>Alice: Thanks for asking
|
||||
end
|
||||
{{< /mermaid >}}
|
||||
|
||||
{{< /columns >}}
|
|
@ -0,0 +1,15 @@
|
|||
---
|
||||
bookCollapseSection: true
|
||||
---
|
||||
|
||||
# Section
|
||||
|
||||
Section renders pages in section as definition list, using title and description.
|
||||
|
||||
## Example
|
||||
|
||||
```tpl
|
||||
{{</* section */>}}
|
||||
```
|
||||
|
||||
{{<section>}}
|
|
@ -0,0 +1 @@
|
|||
# Page 1
|
|
@ -0,0 +1 @@
|
|||
# Page 2
|
|
@ -0,0 +1,50 @@
|
|||
# Tabs
|
||||
|
||||
Tabs let you organize content by context, for example installation instructions for each supported platform.
|
||||
|
||||
```tpl
|
||||
{{</* tabs "uniqueid" */>}}
|
||||
{{</* tab "MacOS" */>}} # MacOS Content {{</* /tab */>}}
|
||||
{{</* tab "Linux" */>}} # Linux Content {{</* /tab */>}}
|
||||
{{</* tab "Windows" */>}} # Windows Content {{</* /tab */>}}
|
||||
{{</* /tabs */>}}
|
||||
```
|
||||
|
||||
## Example
|
||||
|
||||
{{< tabs "uniqueid" >}}
|
||||
{{< tab "MacOS" >}}
|
||||
# MacOS
|
||||
|
||||
This is tab **MacOS** content.
|
||||
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
protulit, sed sed aere valvis inhaesuro Pallas animam: qui _quid_, ignes.
|
||||
Miseratus fonte Ditis conubia.
|
||||
{{< /tab >}}
|
||||
|
||||
{{< tab "Linux" >}}
|
||||
|
||||
# Linux
|
||||
|
||||
This is tab **Linux** content.
|
||||
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
protulit, sed sed aere valvis inhaesuro Pallas animam: qui _quid_, ignes.
|
||||
Miseratus fonte Ditis conubia.
|
||||
{{< /tab >}}
|
||||
|
||||
{{< tab "Windows" >}}
|
||||
|
||||
# Windows
|
||||
|
||||
This is tab **Windows** content.
|
||||
|
||||
Lorem markdownum insigne. Olympo signis Delphis! Retexi Nereius nova develat
|
||||
stringit, frustra Saturnius uteroque inter! Oculis non ritibus Telethusa
|
||||
protulit, sed sed aere valvis inhaesuro Pallas animam: qui _quid_, ignes.
|
||||
Miseratus fonte Ditis conubia.
|
||||
{{< /tab >}}
|
||||
{{< /tabs >}}
|
|
@ -0,0 +1,22 @@
|
|||
---
|
||||
headless: true
|
||||
---
|
||||
|
||||
- [**Example Site**]({{< relref "/docs/example" >}})
|
||||
- [Table of Contents]({{< relref "/docs/example/table-of-contents" >}})
|
||||
- [With ToC]({{< relref "/docs/example/table-of-contents/with-toc" >}})
|
||||
- [Without ToC]({{< relref "/docs/example/table-of-contents/without-toc" >}})
|
||||
- [Collapsed]({{< relref "/docs/example/collapsed" >}})
|
||||
- [3rd]({{< relref "/docs/example/collapsed/3rd-level" >}})
|
||||
- [4th]({{< relref "/docs/example/collapsed/3rd-level/4th-level" >}})
|
||||
<br />
|
||||
|
||||
- **Shortcodes**
|
||||
- [Buttons]({{< relref "/docs/shortcodes/buttons" >}})
|
||||
- [Columns]({{< relref "/docs/shortcodes/columns" >}})
|
||||
- [Expand]({{< relref "/docs/shortcodes/expand" >}})
|
||||
- [Hints]({{< relref "/docs/shortcodes/hints" >}})
|
||||
- [Katex]({{< relref "/docs/shortcodes/katex" >}})
|
||||
- [Mermaid]({{< relref "/docs/shortcodes/mermaid" >}})
|
||||
- [Tabs]({{< relref "/docs/shortcodes/tabs" >}})
|
||||
<br />
|
|
@ -0,0 +1,7 @@
|
|||
---
|
||||
menu:
|
||||
after:
|
||||
name: blog
|
||||
weight: 5
|
||||
title: Blog
|
||||
---
|
|
@ -0,0 +1,344 @@
|
|||
+++
|
||||
title = "(Hu)go Template Primer"
|
||||
description = ""
|
||||
tags = [
|
||||
"go",
|
||||
"golang",
|
||||
"templates",
|
||||
"themes",
|
||||
"development",
|
||||
]
|
||||
date = "2014-04-02"
|
||||
categories = [
|
||||
"Development",
|
||||
"golang",
|
||||
]
|
||||
menu = "main"
|
||||
+++
|
||||
|
||||
Hugo uses the excellent [Go][] [html/template][gohtmltemplate] library for
|
||||
its template engine. It is an extremely lightweight engine that provides a very
|
||||
small amount of logic. In our experience that it is just the right amount of
|
||||
logic to be able to create a good static website. If you have used other
|
||||
template systems from different languages or frameworks you will find a lot of
|
||||
similarities in Go templates.
|
||||
|
||||
This document is a brief primer on using Go templates. The [Go docs][gohtmltemplate]
|
||||
provide more details.
|
||||
|
||||
## Introduction to Go Templates
|
||||
|
||||
Go templates provide an extremely simple template language. It adheres to the
|
||||
belief that only the most basic of logic belongs in the template or view layer.
|
||||
One consequence of this simplicity is that Go templates parse very quickly.
|
||||
|
||||
A unique characteristic of Go templates is they are content aware. Variables and
|
||||
content will be sanitized depending on the context of where they are used. More
|
||||
details can be found in the [Go docs][gohtmltemplate].
|
||||
|
||||
## Basic Syntax
|
||||
|
||||
Golang templates are HTML files with the addition of variables and
|
||||
functions.
|
||||
|
||||
**Go variables and functions are accessible within {{ }}**
|
||||
|
||||
Accessing a predefined variable "foo":
|
||||
|
||||
{{ foo }}
|
||||
|
||||
**Parameters are separated using spaces**
|
||||
|
||||
Calling the add function with input of 1, 2:
|
||||
|
||||
{{ add 1 2 }}
|
||||
|
||||
**Methods and fields are accessed via dot notation**
|
||||
|
||||
Accessing the Page Parameter "bar"
|
||||
|
||||
{{ .Params.bar }}
|
||||
|
||||
**Parentheses can be used to group items together**
|
||||
|
||||
{{ if or (isset .Params "alt") (isset .Params "caption") }} Caption {{ end }}
|
||||
|
||||
|
||||
## Variables
|
||||
|
||||
Each Go template has a struct (object) made available to it. In hugo each
|
||||
template is passed either a page or a node struct depending on which type of
|
||||
page you are rendering. More details are available on the
|
||||
[variables](/layout/variables) page.
|
||||
|
||||
A variable is accessed by referencing the variable name.
|
||||
|
||||
<title>{{ .Title }}</title>
|
||||
|
||||
Variables can also be defined and referenced.
|
||||
|
||||
{{ $address := "123 Main St."}}
|
||||
{{ $address }}
|
||||
|
||||
|
||||
## Functions
|
||||
|
||||
Go template ship with a few functions which provide basic functionality. The Go
|
||||
template system also provides a mechanism for applications to extend the
|
||||
available functions with their own. [Hugo template
|
||||
functions](/layout/functions) provide some additional functionality we believe
|
||||
are useful for building websites. Functions are called by using their name
|
||||
followed by the required parameters separated by spaces. Template
|
||||
functions cannot be added without recompiling hugo.
|
||||
|
||||
**Example:**
|
||||
|
||||
{{ add 1 2 }}
|
||||
|
||||
## Includes
|
||||
|
||||
When including another template you will pass to it the data it will be
|
||||
able to access. To pass along the current context please remember to
|
||||
include a trailing dot. The templates location will always be starting at
|
||||
the /layout/ directory within Hugo.
|
||||
|
||||
**Example:**
|
||||
|
||||
{{ template "chrome/header.html" . }}
|
||||
|
||||
|
||||
## Logic
|
||||
|
||||
Go templates provide the most basic iteration and conditional logic.
|
||||
|
||||
### Iteration
|
||||
|
||||
Just like in Go, the Go templates make heavy use of range to iterate over
|
||||
a map, array or slice. The following are different examples of how to use
|
||||
range.
|
||||
|
||||
**Example 1: Using Context**
|
||||
|
||||
{{ range array }}
|
||||
{{ . }}
|
||||
{{ end }}
|
||||
|
||||
**Example 2: Declaring value variable name**
|
||||
|
||||
{{range $element := array}}
|
||||
{{ $element }}
|
||||
{{ end }}
|
||||
|
||||
**Example 2: Declaring key and value variable name**
|
||||
|
||||
{{range $index, $element := array}}
|
||||
{{ $index }}
|
||||
{{ $element }}
|
||||
{{ end }}
|
||||
|
||||
### Conditionals
|
||||
|
||||
If, else, with, or, & and provide the framework for handling conditional
|
||||
logic in Go Templates. Like range, each statement is closed with `end`.
|
||||
|
||||
|
||||
Go Templates treat the following values as false:
|
||||
|
||||
* false
|
||||
* 0
|
||||
* any array, slice, map, or string of length zero
|
||||
|
||||
**Example 1: If**
|
||||
|
||||
{{ if isset .Params "title" }}<h4>{{ index .Params "title" }}</h4>{{ end }}
|
||||
|
||||
**Example 2: If -> Else**
|
||||
|
||||
{{ if isset .Params "alt" }}
|
||||
{{ index .Params "alt" }}
|
||||
{{else}}
|
||||
{{ index .Params "caption" }}
|
||||
{{ end }}
|
||||
|
||||
**Example 3: And & Or**
|
||||
|
||||
{{ if and (or (isset .Params "title") (isset .Params "caption")) (isset .Params "attr")}}
|
||||
|
||||
**Example 4: With**
|
||||
|
||||
An alternative way of writing "if" and then referencing the same value
|
||||
is to use "with" instead. With rebinds the context `.` within its scope,
|
||||
and skips the block if the variable is absent.
|
||||
|
||||
The first example above could be simplified as:
|
||||
|
||||
{{ with .Params.title }}<h4>{{ . }}</h4>{{ end }}
|
||||
|
||||
**Example 5: If -> Else If**
|
||||
|
||||
{{ if isset .Params "alt" }}
|
||||
{{ index .Params "alt" }}
|
||||
{{ else if isset .Params "caption" }}
|
||||
{{ index .Params "caption" }}
|
||||
{{ end }}
|
||||
|
||||
## Pipes
|
||||
|
||||
One of the most powerful components of Go templates is the ability to
|
||||
stack actions one after another. This is done by using pipes. Borrowed
|
||||
from unix pipes, the concept is simple, each pipeline's output becomes the
|
||||
input of the following pipe.
|
||||
|
||||
Because of the very simple syntax of Go templates, the pipe is essential
|
||||
to being able to chain together function calls. One limitation of the
|
||||
pipes is that they only can work with a single value and that value
|
||||
becomes the last parameter of the next pipeline.
|
||||
|
||||
A few simple examples should help convey how to use the pipe.
|
||||
|
||||
**Example 1 :**
|
||||
|
||||
{{ if eq 1 1 }} Same {{ end }}
|
||||
|
||||
is the same as
|
||||
|
||||
{{ eq 1 1 | if }} Same {{ end }}
|
||||
|
||||
It does look odd to place the if at the end, but it does provide a good
|
||||
illustration of how to use the pipes.
|
||||
|
||||
**Example 2 :**
|
||||
|
||||
{{ index .Params "disqus_url" | html }}
|
||||
|
||||
Access the page parameter called "disqus_url" and escape the HTML.
|
||||
|
||||
**Example 3 :**
|
||||
|
||||
{{ if or (or (isset .Params "title") (isset .Params "caption")) (isset .Params "attr")}}
|
||||
Stuff Here
|
||||
{{ end }}
|
||||
|
||||
Could be rewritten as
|
||||
|
||||
{{ isset .Params "caption" | or isset .Params "title" | or isset .Params "attr" | if }}
|
||||
Stuff Here
|
||||
{{ end }}
|
||||
|
||||
|
||||
## Context (aka. the dot)
|
||||
|
||||
The most easily overlooked concept to understand about Go templates is that {{ . }}
|
||||
always refers to the current context. In the top level of your template this
|
||||
will be the data set made available to it. Inside of a iteration it will have
|
||||
the value of the current item. When inside of a loop the context has changed. .
|
||||
will no longer refer to the data available to the entire page. If you need to
|
||||
access this from within the loop you will likely want to set it to a variable
|
||||
instead of depending on the context.
|
||||
|
||||
**Example:**
|
||||
|
||||
{{ $title := .Site.Title }}
|
||||
{{ range .Params.tags }}
|
||||
<li> <a href="{{ $baseurl }}/tags/{{ . | urlize }}">{{ . }}</a> - {{ $title }} </li>
|
||||
{{ end }}
|
||||
|
||||
Notice how once we have entered the loop the value of {{ . }} has changed. We
|
||||
have defined a variable outside of the loop so we have access to it from within
|
||||
the loop.
|
||||
|
||||
# Hugo Parameters
|
||||
|
||||
Hugo provides the option of passing values to the template language
|
||||
through the site configuration (for sitewide values), or through the meta
|
||||
data of each specific piece of content. You can define any values of any
|
||||
type (supported by your front matter/config format) and use them however
|
||||
you want to inside of your templates.
|
||||
|
||||
|
||||
## Using Content (page) Parameters
|
||||
|
||||
In each piece of content you can provide variables to be used by the
|
||||
templates. This happens in the [front matter](/content/front-matter).
|
||||
|
||||
An example of this is used in this documentation site. Most of the pages
|
||||
benefit from having the table of contents provided. Sometimes the TOC just
|
||||
doesn't make a lot of sense. We've defined a variable in our front matter
|
||||
of some pages to turn off the TOC from being displayed.
|
||||
|
||||
Here is the example front matter:
|
||||
|
||||
```
|
||||
---
|
||||
title: "Permalinks"
|
||||
date: "2013-11-18"
|
||||
aliases:
|
||||
- "/doc/permalinks/"
|
||||
groups: ["extras"]
|
||||
groups_weight: 30
|
||||
notoc: true
|
||||
---
|
||||
```
|
||||
|
||||
Here is the corresponding code inside of the template:
|
||||
|
||||
{{ if not .Params.notoc }}
|
||||
<div id="toc" class="well col-md-4 col-sm-6">
|
||||
{{ .TableOfContents }}
|
||||
</div>
|
||||
{{ end }}
|
||||
|
||||
|
||||
|
||||
## Using Site (config) Parameters
|
||||
In your top-level configuration file (eg, `config.yaml`) you can define site
|
||||
parameters, which are values which will be available to you in chrome.
|
||||
|
||||
For instance, you might declare:
|
||||
|
||||
```yaml
|
||||
params:
|
||||
CopyrightHTML: "Copyright © 2013 John Doe. All Rights Reserved."
|
||||
TwitterUser: "spf13"
|
||||
SidebarRecentLimit: 5
|
||||
```
|
||||
|
||||
Within a footer layout, you might then declare a `<footer>` which is only
|
||||
provided if the `CopyrightHTML` parameter is provided, and if it is given,
|
||||
you would declare it to be HTML-safe, so that the HTML entity is not escaped
|
||||
again. This would let you easily update just your top-level config file each
|
||||
January 1st, instead of hunting through your templates.
|
||||
|
||||
```
|
||||
{{if .Site.Params.CopyrightHTML}}<footer>
|
||||
<div class="text-center">{{.Site.Params.CopyrightHTML | safeHtml}}</div>
|
||||
</footer>{{end}}
|
||||
```
|
||||
|
||||
An alternative way of writing the "if" and then referencing the same value
|
||||
is to use "with" instead. With rebinds the context `.` within its scope,
|
||||
and skips the block if the variable is absent:
|
||||
|
||||
```
|
||||
{{with .Site.Params.TwitterUser}}<span class="twitter">
|
||||
<a href="https://twitter.com/{{.}}" rel="author">
|
||||
<img src="/images/twitter.png" width="48" height="48" title="Twitter: {{.}}"
|
||||
alt="Twitter"></a>
|
||||
</span>{{end}}
|
||||
```
|
||||
|
||||
Finally, if you want to pull "magic constants" out of your layouts, you can do
|
||||
so, such as in this example:
|
||||
|
||||
```
|
||||
<nav class="recent">
|
||||
<h1>Recent Posts</h1>
|
||||
<ul>{{range first .Site.Params.SidebarRecentLimit .Site.Recent}}
|
||||
<li><a href="{{.RelPermalink}}">{{.Title}}</a></li>
|
||||
{{end}}</ul>
|
||||
</nav>
|
||||
```
|
||||
|
||||
|
||||
[go]: https://golang.org/
|
||||
[gohtmltemplate]: https://golang.org/pkg/html/template/
|
|
@ -0,0 +1,89 @@
|
|||
+++
|
||||
title = "Getting Started with Hugo"
|
||||
description = ""
|
||||
tags = [
|
||||
"go",
|
||||
"golang",
|
||||
"hugo",
|
||||
"development",
|
||||
]
|
||||
date = "2014-04-02"
|
||||
categories = [
|
||||
"Development",
|
||||
"golang",
|
||||
]
|
||||
menu = "main"
|
||||
+++
|
||||
|
||||
## Step 1. Install Hugo
|
||||
|
||||
Go to [Hugo releases](https://github.com/spf13/hugo/releases) and download the
|
||||
appropriate version for your OS and architecture.
|
||||
|
||||
Save it somewhere specific as we will be using it in the next step.
|
||||
|
||||
More complete instructions are available at [Install Hugo](https://gohugo.io/getting-started/installing/)
|
||||
|
||||
## Step 2. Build the Docs
|
||||
|
||||
Hugo has its own example site which happens to also be the documentation site
|
||||
you are reading right now.
|
||||
|
||||
Follow the following steps:
|
||||
|
||||
1. Clone the [Hugo repository](http://github.com/spf13/hugo)
|
||||
2. Go into the repo
|
||||
3. Run hugo in server mode and build the docs
|
||||
4. Open your browser to http://localhost:1313
|
||||
|
||||
Corresponding pseudo commands:
|
||||
|
||||
git clone https://github.com/spf13/hugo
|
||||
cd hugo
|
||||
/path/to/where/you/installed/hugo server --source=./docs
|
||||
> 29 pages created
|
||||
> 0 tags index created
|
||||
> in 27 ms
|
||||
> Web Server is available at http://localhost:1313
|
||||
> Press ctrl+c to stop
|
||||
|
||||
Once you've gotten here, follow along the rest of this page on your local build.
|
||||
|
||||
## Step 3. Change the docs site
|
||||
|
||||
Stop the Hugo process by hitting Ctrl+C.
|
||||
|
||||
Now we are going to run hugo again, but this time with hugo in watch mode.
|
||||
|
||||
/path/to/hugo/from/step/1/hugo server --source=./docs --watch
|
||||
> 29 pages created
|
||||
> 0 tags index created
|
||||
> in 27 ms
|
||||
> Web Server is available at http://localhost:1313
|
||||
> Watching for changes in /Users/spf13/Code/hugo/docs/content
|
||||
> Press ctrl+c to stop
|
||||
|
||||
|
||||
Open your [favorite editor](http://vim.spf13.com) and change one of the source
|
||||
content pages. How about changing this very file to *fix the typo*. How about changing this very file to *fix the typo*.
|
||||
|
||||
Content files are found in `docs/content/`. Unless otherwise specified, files
|
||||
are located at the same relative location as the url, in our case
|
||||
`docs/content/overview/quickstart.md`.
|
||||
|
||||
Change and save this file.. Notice what happened in your terminal.
|
||||
|
||||
> Change detected, rebuilding site
|
||||
|
||||
> 29 pages created
|
||||
> 0 tags index created
|
||||
> in 26 ms
|
||||
|
||||
Refresh the browser and observe that the typo is now fixed.
|
||||
|
||||
Notice how quick that was. Try to refresh the site before it's finished building. I double dare you.
|
||||
Having nearly instant feedback enables you to have your creativity flow without waiting for long builds.
|
||||
|
||||
## Step 4. Have fun
|
||||
|
||||
The best way to learn something is to play with it.
|
|
@ -0,0 +1,156 @@
|
|||
---
|
||||
date: 2014-03-10
|
||||
linktitle: Migrating from Jekyll
|
||||
menu:
|
||||
main:
|
||||
parent: tutorials
|
||||
prev: /tutorials/mathjax
|
||||
title: Migrate to Hugo from Jekyll
|
||||
weight: 10
|
||||
---
|
||||
|
||||
## Move static content to `static`
|
||||
Jekyll has a rule that any directory not starting with `_` will be copied as-is to the `_site` output. Hugo keeps all static content under `static`. You should therefore move it all there.
|
||||
With Jekyll, something that looked like
|
||||
|
||||
▾ <root>/
|
||||
▾ images/
|
||||
logo.png
|
||||
|
||||
should become
|
||||
|
||||
▾ <root>/
|
||||
▾ static/
|
||||
▾ images/
|
||||
logo.png
|
||||
|
||||
Additionally, you'll want any files that should reside at the root (such as `CNAME`) to be moved to `static`.
|
||||
|
||||
## Create your Hugo configuration file
|
||||
Hugo can read your configuration as JSON, YAML or TOML. Hugo supports parameters custom configuration too. Refer to the [Hugo configuration documentation](/overview/configuration/) for details.
|
||||
|
||||
## Set your configuration publish folder to `_site`
|
||||
The default is for Jekyll to publish to `_site` and for Hugo to publish to `public`. If, like me, you have [`_site` mapped to a git submodule on the `gh-pages` branch](http://blog.blindgaenger.net/generate_github_pages_in_a_submodule.html), you'll want to do one of two alternatives:
|
||||
|
||||
1. Change your submodule to point to map `gh-pages` to public instead of `_site` (recommended).
|
||||
|
||||
git submodule deinit _site
|
||||
git rm _site
|
||||
git submodule add -b gh-pages git@github.com:your-username/your-repo.git public
|
||||
|
||||
2. Or, change the Hugo configuration to use `_site` instead of `public`.
|
||||
|
||||
{
|
||||
..
|
||||
"publishdir": "_site",
|
||||
..
|
||||
}
|
||||
|
||||
## Convert Jekyll templates to Hugo templates
|
||||
That's the bulk of the work right here. The documentation is your friend. You should refer to [Jekyll's template documentation](http://jekyllrb.com/docs/templates/) if you need to refresh your memory on how you built your blog and [Hugo's template](/layout/templates/) to learn Hugo's way.
|
||||
|
||||
As a single reference data point, converting my templates for [heyitsalex.net](http://heyitsalex.net/) took me no more than a few hours.
|
||||
|
||||
## Convert Jekyll plugins to Hugo shortcodes
|
||||
Jekyll has [plugins](http://jekyllrb.com/docs/plugins/); Hugo has [shortcodes](/doc/shortcodes/). It's fairly trivial to do a port.
|
||||
|
||||
### Implementation
|
||||
As an example, I was using a custom [`image_tag`](https://github.com/alexandre-normand/alexandre-normand/blob/74bb12036a71334fdb7dba84e073382fc06908ec/_plugins/image_tag.rb) plugin to generate figures with caption when running Jekyll. As I read about shortcodes, I found Hugo had a nice built-in shortcode that does exactly the same thing.
|
||||
|
||||
Jekyll's plugin:
|
||||
|
||||
module Jekyll
|
||||
class ImageTag < Liquid::Tag
|
||||
@url = nil
|
||||
@caption = nil
|
||||
@class = nil
|
||||
@link = nil
|
||||
// Patterns
|
||||
IMAGE_URL_WITH_CLASS_AND_CAPTION =
|
||||
IMAGE_URL_WITH_CLASS_AND_CAPTION_AND_LINK = /(\w+)(\s+)((https?:\/\/|\/)(\S+))(\s+)"(.*?)"(\s+)->((https?:\/\/|\/)(\S+))(\s*)/i
|
||||
IMAGE_URL_WITH_CAPTION = /((https?:\/\/|\/)(\S+))(\s+)"(.*?)"/i
|
||||
IMAGE_URL_WITH_CLASS = /(\w+)(\s+)((https?:\/\/|\/)(\S+))/i
|
||||
IMAGE_URL = /((https?:\/\/|\/)(\S+))/i
|
||||
def initialize(tag_name, markup, tokens)
|
||||
super
|
||||
if markup =~ IMAGE_URL_WITH_CLASS_AND_CAPTION_AND_LINK
|
||||
@class = $1
|
||||
@url = $3
|
||||
@caption = $7
|
||||
@link = $9
|
||||
elsif markup =~ IMAGE_URL_WITH_CLASS_AND_CAPTION
|
||||
@class = $1
|
||||
@url = $3
|
||||
@caption = $7
|
||||
elsif markup =~ IMAGE_URL_WITH_CAPTION
|
||||
@url = $1
|
||||
@caption = $5
|
||||
elsif markup =~ IMAGE_URL_WITH_CLASS
|
||||
@class = $1
|
||||
@url = $3
|
||||
elsif markup =~ IMAGE_URL
|
||||
@url = $1
|
||||
end
|
||||
end
|
||||
def render(context)
|
||||
if @class
|
||||
source = "<figure class='#{@class}'>"
|
||||
else
|
||||
source = "<figure>"
|
||||
end
|
||||
if @link
|
||||
source += "<a href=\"#{@link}\">"
|
||||
end
|
||||
source += "<img src=\"#{@url}\">"
|
||||
if @link
|
||||
source += "</a>"
|
||||
end
|
||||
source += "<figcaption>#{@caption}</figcaption>" if @caption
|
||||
source += "</figure>"
|
||||
source
|
||||
end
|
||||
end
|
||||
end
|
||||
Liquid::Template.register_tag('image', Jekyll::ImageTag)
|
||||
|
||||
is written as this Hugo shortcode:
|
||||
|
||||
<!-- image -->
|
||||
<figure {{ with .Get "class" }}class="{{.}}"{{ end }}>
|
||||
{{ with .Get "link"}}<a href="{{.}}">{{ end }}
|
||||
<img src="{{ .Get "src" }}" {{ if or (.Get "alt") (.Get "caption") }}alt="{{ with .Get "alt"}}{{.}}{{else}}{{ .Get "caption" }}{{ end }}"{{ end }} />
|
||||
{{ if .Get "link"}}</a>{{ end }}
|
||||
{{ if or (or (.Get "title") (.Get "caption")) (.Get "attr")}}
|
||||
<figcaption>{{ if isset .Params "title" }}
|
||||
{{ .Get "title" }}{{ end }}
|
||||
{{ if or (.Get "caption") (.Get "attr")}}<p>
|
||||
{{ .Get "caption" }}
|
||||
{{ with .Get "attrlink"}}<a href="{{.}}"> {{ end }}
|
||||
{{ .Get "attr" }}
|
||||
{{ if .Get "attrlink"}}</a> {{ end }}
|
||||
</p> {{ end }}
|
||||
</figcaption>
|
||||
{{ end }}
|
||||
</figure>
|
||||
<!-- image -->
|
||||
|
||||
### Usage
|
||||
I simply changed:
|
||||
|
||||
{% image full http://farm5.staticflickr.com/4136/4829260124_57712e570a_o_d.jpg "One of my favorite touristy-type photos. I secretly waited for the good light while we were "having fun" and took this. Only regret: a stupid pole in the top-left corner of the frame I had to clumsily get rid of at post-processing." ->http://www.flickr.com/photos/alexnormand/4829260124/in/set-72157624547713078/ %}
|
||||
|
||||
to this (this example uses a slightly extended version named `fig`, different than the built-in `figure`):
|
||||
|
||||
{{%/* fig class="full" src="http://farm5.staticflickr.com/4136/4829260124_57712e570a_o_d.jpg" title="One of my favorite touristy-type photos. I secretly waited for the good light while we were having fun and took this. Only regret: a stupid pole in the top-left corner of the frame I had to clumsily get rid of at post-processing." link="http://www.flickr.com/photos/alexnormand/4829260124/in/set-72157624547713078/" */%}}
|
||||
|
||||
As a bonus, the shortcode named parameters are, arguably, more readable.
|
||||
|
||||
## Finishing touches
|
||||
### Fix content
|
||||
Depending on the amount of customization that was done with each post with Jekyll, this step will require more or less effort. There are no hard and fast rules here except that `hugo server --watch` is your friend. Test your changes and fix errors as needed.
|
||||
|
||||
### Clean up
|
||||
You'll want to remove the Jekyll configuration at this point. If you have anything else that isn't used, delete it.
|
||||
|
||||
## A practical example in a diff
|
||||
[Hey, it's Alex](http://heyitsalex.net/) was migrated in less than a _father-with-kids day_ from Jekyll to Hugo. You can see all the changes (and screw-ups) by looking at this [diff](https://github.com/alexandre-normand/alexandre-normand/compare/869d69435bd2665c3fbf5b5c78d4c22759d7613a...b7f6605b1265e83b4b81495423294208cc74d610).
|
|
@ -0,0 +1,43 @@
|
|||
{{ define "main" }}
|
||||
<article class="markdown">
|
||||
{{ printf "# %s" .Params.title | markdownify }}
|
||||
{{ if .Params.name }}
|
||||
{{ printf "# %s" .Params.name | markdownify }}
|
||||
{{ end }}
|
||||
|
||||
{{ if eq .Params.status "raw" }}
|
||||
<img src="https://img.shields.io/badge/status-{{.Params.status}}-lightgrey?style=flat-square" />
|
||||
{{ end }}
|
||||
{{ if eq .Params.status "draft" }}
|
||||
<img src="https://img.shields.io/badge/status-{{.Params.status}}-blue?style=flat-square" />
|
||||
{{ end }}
|
||||
{{ if eq .Params.status "stable" }}
|
||||
<img src="https://img.shields.io/badge/status-{{.Params.status}}-brightgreen?style=flat-square" />
|
||||
{{ end }}
|
||||
{{ if eq .Params.status "deprecated" }}
|
||||
<img src="https://img.shields.io/badge/status-{{.Params.status}}-yellowgreen?style=flat-square" />
|
||||
{{ end }}
|
||||
{{ if eq .Params.status "retired" }}
|
||||
<img src="https://img.shields.io/badge/status-{{.Params.status}}-red?style=flat-square" />
|
||||
{{ end }}
|
||||
|
||||
<ul>
|
||||
<li>Status: {{ .Params.status | markdownify }}</li>
|
||||
<li>Editor: {{ .Params.editor | markdownify }}</li>
|
||||
{{ if .Params.contributors }}
|
||||
<li>Contributors:
|
||||
{{ range $index, $element := .Params.contributors }}
|
||||
{{if $index}},{{end}}
|
||||
{{ . | markdownify }}
|
||||
{{ end }}
|
||||
</li>
|
||||
{{ end }}
|
||||
</ul>
|
||||
|
||||
{{- .Content -}}
|
||||
</article>
|
||||
{{ end }}
|
||||
|
||||
{{ define "toc" }}
|
||||
{{ partial "docs/toc" . }}
|
||||
{{ end }}
|
|
@ -0,0 +1,6 @@
|
|||
[build]
|
||||
command = "hugo"
|
||||
publish = "public"
|
||||
|
||||
[build.environment]
|
||||
HUGO_VERSION = "0.80.0"
|