status-im/specs
2
0
Fork 0
mirror of https://github.com/status-im/specs.git synced 2025-01-27 06:34:58 +00:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into eip-1459

Browse Source
This commit is contained in:
decanus 2020-05-06 16:47:07 +02:00
commit 1bebe99593
No known key found for this signature in database
GPG Key ID: 3730AAF5D6589867
27 changed files with 1049 additions and 700 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
.remarkrc Normal file
View File

@ -0,0 +1,30 @@
{
"presets": ["lint-recommended", "lint-consistent"],
"plugins": {
"remark-lint": {
"unordered-list-marker-style": "consistent",
"list-item-bullet-indent": true,
"list-item-indent": false,
"list-item-spacing": false,
"no-html": false,
"maximum-line-length": false,
"no-file-name-mixed-case": false,
"heading-increment": false,
"no-multiple-toplevel-headings": true,
"no-consecutive-blank-lines": false,
"maximum-heading-length": 300,
"no-heading-punctuation": false,
"no-duplicate-headings": false,
"emphasis-marker": "*",
"no-tabs": true,
"blockquote-indentation": false,
"strong-marker": "*"
}
},
"settings": {
"bullet": "*",
"listItemIndent": "2",
"strong": "*",
"emphasis": "*"
}
}

15
.travis.yml Normal file
View File

@ -0,0 +1,15 @@
sudo: required
dist: trusty
language: node_js
node_js:
- "8"
script:
- npm run lint
notifications:
email: false

1
CNAME Normal file
View File

@ -0,0 +1 @@
specs.status.im

43
README.md
View File

@ -1,32 +1,34 @@
---
layout: default
permalink: /
nav_exclude: true
---
# Specifications for Status clients
This repository contains a list of specifications for implementing Status and
its various capabilities.
## Current state
## How to contribute
As of August 2019, we are currently in the process of documenting current
specifications. We are als implementing an isolated reference library for them.
These specifications are expected to be frozen at the Status V1 launch and be
used as a reference point for client implementers and security audits.
1. Create an issue for a new Status Improvement Proposal (SIP) or some bug that you'd like to address
2. Create a corresponding PR and ping some exisiting SIP editors for review
If you need help, ask in #protocol at Status / Discord.
## Spec lifecycle
Every spec has its own lifecycle that shows its maturity. We indicate this in a similar fashion to [COSS Lifecycle](https://rfc.unprotocols.org/spec:2/COSS/):
![](assets/lifecycle.png)
At present (March 30, 2020) this means stable specs are what is in v1 of the Status App. Drafts and raw are work in progress specs.
## Status Improvement Proposals (SIPs)
### Accepted
The main specification for writing a Status client is [1/CLIENT](https://specs.status.im/spec/1).
No accepted SIPs right now.
### Draft
The following SIPs are under consideration for standardization.
- [Status Client Specification](status-client-spec.md). The main specification for writing a Status client. **Start here**
- [Status Secure Transport Specification](status-secure-transport-spec.md). How Status provide a secure transport with conversational security properties.
- [Status Payload Specification](status-payloads-spec.md). What the message payloads look like.
- [Status Account Specification](status-account-spec.md). What a Status account is and how trust is established.
- [Status Whisper Usage Specification](status-whisper-usage-spec.md). How we use Whisper to do routing, metadata protection and provide 1:1/group/public chat.
- [Status Whisper Mailserver Specification](status-whisper-mailserver-spec.md). How we use Whisper mailservers to provide offline inboxing.
- [Status EIPs Standards](status-EIPs.md). Ethereum Improvement Proposals used in Status.
For all full index of all specs, see [specs.status.im](https://specs.status.im/), especially stable specs.
## Protocol Research
@ -34,5 +36,4 @@ These are protocols that are currently being researched. These are designed to
be useful outside of Status as well. To the extent that these protocols are used
within Status clients, they will show up as SIPs in the future.
To see more on this, please visit the current home: [vac
protocol](https://specs.vac.dev).
To see more on this, please visit the current home: [vac protocol](https://specs.vac.dev).

10
_config.yml
View File

@ -1,2 +1,10 @@
theme: jekyll-theme-minimal
title: "Status Specification"
remote_theme: pmarsceill/just-the-docs
color_scheme: "light"
search_enabled: true
url: "https://specs.status.im"
exclude: ["node_modules/", "*.gemspec", "*.gem", "Gemfile", "Gemfile.lock", "package.json", "package-lock.json", "script/", "LICENSE.txt", "lib/", "bin/", "Rakefile"]
plugins:
- jekyll-sitemap

BIN
assets/lifecycle.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 42 KiB

75
default.html Normal file
View File

@ -0,0 +1,75 @@
<!DOCTYPE html>
<html lang="en-us">
{% include head.html %}
<div class="page-wrap">
<div class="side-bar">
<a href="{{ site.url }}{{ site.baseurl }}" class="site-title fs-6 lh-tight">{{ site.title }}</a>
<span class="fs-3"><button class="js-main-nav-trigger navigation-list-toggle btn btn-outline" type="button" data-text-toggle="Hide">Menu</button></span>
<div class="navigation main-nav js-main-nav">
{% include nav.html %}
</div>
<footer role="contentinfo" class="site-footer">
<p class="text-small text-grey-dk-000 mb-0">This site uses <a href="https://github.com/pmarsceill/just-the-docs">Just the Docs</a>, a documentation theme for Jekyll.</p>
</footer>
</div>
<div class="main-content-wrap">
<div class="page-header">
<div class="main-content">
{% if site.search_enabled != nil %}
<div class="search js-search">
<div class="search-input-wrap">
<input type="text" class="js-search-input search-input" placeholder="Search {{ site.title }}" aria-label="Search {{ site.title }}" autocomplete="off">
<svg width="14" height="14" viewBox="0 0 28 28" xmlns="http://www.w3.org/2000/svg" class="search-icon"><title>Search</title><g fill-rule="nonzero"><path d="M17.332 20.735c-5.537 0-10-4.6-10-10.247 0-5.646 4.463-10.247 10-10.247 5.536 0 10 4.601 10 10.247s-4.464 10.247-10 10.247zm0-4c3.3 0 6-2.783 6-6.247 0-3.463-2.7-6.247-6-6.247s-6 2.784-6 6.247c0 3.464 2.7 6.247 6 6.247z"/><path d="M11.672 13.791L.192 25.271 3.02 28.1 14.5 16.62z"/></g></svg>
</div>
<div class="js-search-results search-results-wrap"></div>
</div>
{% endif %}
{% if site.aux_links != nil %}
<ul class="list-style-none text-small mt-md-1 mb-md-1 pb-4 pb-md-0 js-aux-nav aux-nav">
{% for link in site.aux_links %}
<li class="d-inline-block my-0{% unless forloop.last %} mr-2{% endunless %}"><a href="{{ link.last }}">{{ link.first }}</a></li>
{% endfor %}
</ul>
{% endif %}
</div>
</div>
<div class="main-content">
{% unless page.url == "/" %}
{% if page.parent %}
<nav class="breadcrumb-nav">
<ol class="breadcrumb-nav-list">
{% if page.grand_parent %}
<li class="breadcrumb-nav-list-item"><a href="{{ first_level_url }}">{{ page.grand_parent }}</a></li>
<li class="breadcrumb-nav-list-item"><a href="{{ second_level_url }}">{{ page.parent }}</a></li>
{% else %}
<li class="breadcrumb-nav-list-item"><a href="{{ first_level_url }}">{{ page.parent }}</a></li>
{% endif %}
<li class="breadcrumb-nav-list-item"><span>{{ page.title }}</span></li>
</ol>
</nav>
{% endif %}
{% endunless %}
<div class="page-content">
{{ content }}
{% if page.has_children == true %}
<hr>
<h2 class="text-delta">Table of contents</h2>
{% assign children_list = site.pages | sort:"nav_order" %}
<ul>
{% for child in children_list %}
{% if child.parent == page.title and child.title != page.title %}
<li>
<a href="{{ child.url | absolute_url }}">{{ child.title }}</a>
</li>
{% endif %}
{% endfor %}
</ul>
{% endif %}
</div>
</div>
</div>
</div>
</html>

157
docs/draft/7-group-chat.md Normal file
View File

@ -0,0 +1,157 @@
---
permalink: /spec/7
parent: Draft specs
title: 7/GROUP-CHAT
---
# 7/GROUP-CHAT
> Version: 0.1
>
> Status: Draft
>
> Authors: Andrea Maria Piana <andreap@status.im>
>
## Table of Contents
- [Abstract](#abstract)
- [Membership updates](#membership-updates)
- [Chat ID](#chat-id)
- [Signature](#signature)
- [Group membership event](#group-membership-event)
- [chat-created](#chat-created)
- [name-changed](#name-changed)
- [members-added](#members-added)
- [members-joined](#members-joined)
- [admins-added](#admins-added)
- [members-removed](#members-removed)
- [admin-removed](#admin-removed)
## Abstract
This documents describes the group chat protocol used by the status application. Pairwise encryption is used among member so a message is exchanged between each participants, similarly to a one-to-one message.
## Membership updates
Membership updates messages are used to propagate group chat membership changes. The protobuf format is described in the [6/PAYLOADS](https://specs.status.im/spec/6). Here we will be describing each specific field.
The protobuf messages are:
```protobuf
// MembershipUpdateMessage is a message used to propagate information
// about group membership changes.
message MembershipUpdateMessage {
// The chat id of the private group chat
string chat_id = 1;
// A list of events for this group chat, first 65 bytes are the signature, then is a
// protobuf encoded MembershipUpdateEvent
repeated bytes events = 2;
// An optional chat message
ChatMessage message = 3;
}
message MembershipUpdateEvent {
// Lamport timestamp of the event as described in [Status Payload Specs](status-payload-specs.md#clock-vs-timestamp-and-message-ordering)
uint64 clock = 1;
// List of public keys of the targets of the action
repeated string members = 2;
// Name of the chat for the CHAT_CREATED/NAME_CHANGED event types
string name = 3;
// The type of the event
EventType type = 4;
enum EventType {
UNKNOWN = 0;
CHAT_CREATED = 1; // See [CHAT_CREATED](#chat-created)
NAME_CHANGED = 2; // See [NAME_CHANGED](#name-changed)
MEMBERS_ADDED = 3; // See [MEMBERS_ADDED](#members-added)
MEMBER_JOINED = 4; // See [MEMBER_JOINED](#member-joined)
MEMBER_REMOVED = 5; // See [MEMBER_REMOVED](#member-removed)
ADMINS_ADDED = 6; // See [ADMINS_ADDED](#admins-added)
ADMIN_REMOVED = 7; // See [ADMIN_REMOVED](#admin-removed)
}
}
```
### Payload
`MembershipUpdateMessage`:
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | chat-id | `string` | The chat id of the chat where the change is to take place |
| 2 | events | See details | A list of events that describe the membership changes, in their encoded protobuf form |
| 3 | message | `ChatMessage` | An optional message, described in [Message](#message) |
`MembershipUpdateEvent`:
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | clock | `uint64` | The clock value of the event |
| 2 | members | `[]string` | An optional list of hex encoded (prefixed with `0x`) public keys, the targets of the action |
| 3 | name | `name` | An optional name, for those events that make use of it |
| 4 | type | `EventType` | The type of event sent, described below |
### Chat ID
Each membership update MUST be sent with a corresponding `chatId`.
The format of this chat ID MUST be a string of [UUID](https://tools.ietf.org/html/rfc4122 ), concatenated with the hex-encoded public key of the creator of the chat, joined by `-`. This chatId MUST be validated by all clients, and MUST be discarded if it does not follow these rules.
### Signature
The signature for each event is calculated by encoding each `MembershipUpdateEvent` in its protobuf representation and prepending the bytes of the chatID, lastly the `Keccak256` of the bytes is signed using the private key by the author and added to the `events` field of MembershipUpdateMessage.
### Group membership event
Any group membership event received MUST be verified by calculating the signature as per the method described above.
The author MUST be extracted from it, if the verification fails the event MUST be discarded.
#### CHAT_CREATED
Chat created event is the first event that needs to be sent. Any event with a clock value lower then this MUST be discarded.
Upon receiving this event a client MUST validate the `chatId` provided with the updates and create a chat with identified by `chatId` and named `name`.
#### NAME_CHANGED
A name changed event is used by admins to change the name of the group chat.
Upon receiving this event a client MUST validate the `chatId` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
If the event is valid the chat name SHOULD be changed to `name`.
#### MEMBERS_ADDED
A members added event is used by admins to add members to the chat.
Upon receiving this event a client MUST validate the `chatId` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
If the event is valid a client MUST update the list of members of the chat who have not joined, adding the `members` received.
`members` is an array of hex encoded public keys.
#### MEMBER_JOINED
A members joined event is used by a member of the chat to signal that they want to start receiving messages from this chat.
Upon receiving this event a client MUST validate the `chatId` provided with the updates.
If the event is valid a client MUST update the list of members of the chat who joined, adding the signer. Any `message` sent to the group chat should now include the newly joined member.
#### ADMINS_ADDED
An admins added event is used by admins to add make other admins in the chat.
Upon receiving this event a client MUST validate the `chatId` provided with the updates, MUST ensure the author of the event is an admin of the chat and MUST ensure all `members` are already `members` of the chat, otherwise the event MUST be ignored.
If the event is valid a client MUST update the list of admins of the chat, adding the `members` received.
`members` is an array of hex encoded public keys.
#### MEMBER_REMOVED
A member-removed event is used to leave or kick members of the chat.
Upon receiving this event a client MUST validate the `chatId` provided with the updates, MUST ensure that:
- If the author of the event is an admin, target can only be themselves or a non-admin member.
- If the author of the event is not an admin, the target of the event can only be themselves.
-
If the event is valid a client MUST remove the member from the list of `members`/`admins` of the chat, and no further message should be sent to them.
#### ADMIN_REMOVED
An admin-removed event is used to drop admin privileges.
Upon receiving this event a client MUST validate the `chatId` provided with the updates, MUST ensure that the author of the event is also the target of the event.
If the event is valid a client MUST remove the member from the list of `admins` of the chat.

9
docs/draft/draft.md Normal file
View File

@ -0,0 +1,9 @@
---
layout: default
title: Draft specs
nav_order: 2
has_children: true
permalink: /specs/draft
---
# Draft specifications

9
docs/raw/raw.md Normal file
View File

@ -0,0 +1,9 @@
---
layout: default
title: Raw specs
permanlink: /specs/raw
nav_order: 2
has_children: true
---
# Raw specifications

15
docs/raw/status-blockchain-spec.md Normal file
View File

@ -0,0 +1,15 @@
---
layout: default
nav_exclude: true
parent: Raw specs
---
# Status Blockchain Specification
> Version: 0.1
>
> Status: Draft
>
> Authors: Corey Petty [corey@status.im](mailto:corey@status.im) (alphabetical order)
TODO

14
docs/raw/status-browser-spec.md Normal file
View File

@ -0,0 +1,14 @@
---
nav_exclude: true
parent: Raw specs
---
# Status Browser Specification
> Version: 0.1
>
> Status: Draft
>
> Authors: Corey Petty [corey@status.im](mailto:corey@status.im) (alphabetical order)
TODO

14
docs/raw/status-storage-spec.md Normal file
View File

@ -0,0 +1,14 @@
---
nav_exclude: true
parent: Raw specs
---
# Status Storage Specification
> Version: 0.1
>
> Status: Draft
>
> Authors: Corey Petty [corey@status.im](mailto:corey@status.im) (alphabetical order)
TODO

199
status-client-spec.md → docs/stable/1-client.md
View File

@ -1,8 +1,16 @@
# Status Client Specification
---
permalink: /spec/1
parent: Stable specs
title: 1/CLIENT
---
> Version: 0.1 (Draft)
# 1/CLIENT
> Version: 0.2
>
> Authors: Adam Babik <adam@status.im>, Dean Eigenmann <dean@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
> Status: Stable
>
> Authors: Adam Babik [adam@status.im](mailto:adam@status.im), Andrea Maria Piana [andreap@status.im](mailto:andreap@status.im), Dean Eigenmann [dean@status.im](mailto:dean@status.im), Corey Petty [corey@status.im](mailto:corey@status.im), Oskar Thorén [oskar@status.im](mailto:oskar@status.im) (alphabetical order)
## Abstract
@ -16,75 +24,80 @@ This document consists of two parts. The first outlines the specifications that
have to be implemented in order to be a full Status client. The second gives a design rationale and answers some common questions.
## Table of Contents
- [Status Client Specification](#status-client-specification)
- [Abstract](#abstract)
- [Table of Contents](#table-of-contents)
- [Introduction](#introduction)
- [Protocol layers](#protocol-layers)
- [Components](#components)
- [P2P Overlay](#p2p-overlay)
- [Node discovery and roles](#node-discovery-and-roles)
- [Bootstrapping](#bootstrapping)
- [Discovery](#discovery)
- [Mobile nodes](#mobile-nodes)
- [Transport privacy and Whisper usage](#transport-privacy-and-whisper-usage)
- [Secure Transport](#secure-transport)
- [Data Sync](#data-sync)
- [Payloads and clients](#payloads-and-clients)
- [BIPs and EIPs Standards support](#bips-and-eips-standards-support)
- [Security Considerations](#security-considerations)
- [Censorship-resistance](#censorship-resistance)
- [Design Rationale](#design-rationale)
- [P2P Overlay](#p2p-overlay-1)
- [Why devp2p? Why not use libp2p?](#why-devp2p-why-not-use-libp2p)
- [What about other RLPx subprotocols like LES, and Swarm?](#what-about-other-rlpx-subprotocols-like-les-and-swarm)
- [Why do you use Whisper?](#why-do-you-use-whisper)
- [I heard you were moving away from Whisper?](#i-heard-you-were-moving-away-from-whisper)
- [Why is PoW for Whisper set so low?](#why-is-pow-for-whisper-set-so-low)
- [Why do you not use Discovery v5 for node discovery?](#why-do-you-not-use-discovery-v5-for-node-discovery)
- [I heard something about mailservers being trusted somehow?](#i-heard-something-about-mailservers-being-trusted-somehow)
- [Data sync](#data-sync)
- [Why is MVDS not used for public chats?](#why-is-mvds-not-used-for-public-chats)
- [Footnotes](#footnotes)
- [Appendix A: Security considerations](#appendix-a-security-considerations)
- [Scalability and UX](#scalability-and-ux)
- [Privacy](#privacy)
- [Spam resistance](#spam-resistance)
- [Censorship resistance](#censorship-resistance)
- [Acknowledgements](#acknowledgements)
## Introduction
- [Status Client Specification](#status-client-specification)
- [Abstract](#abstract)
- [Table of Contents](#table-of-contents)
- [Introduction](#introduction)
- [Protocol layers](#protocol-layers)
- [Protobuf](#protobuf)
- [Components](#components)
- [P2P Overlay](#p2p-overlay)
- [Node discovery and roles](#node-discovery-and-roles)
- [Bootstrapping](#bootstrapping)
- [Discovery](#discovery)
- [Mobile nodes](#mobile-nodes)
- [Transport privacy and Whisper usage](#transport-privacy-and-whisper-usage)
- [Secure Transport](#secure-transport)
- [Data Sync](#data-sync)
- [Payloads and clients](#payloads-and-clients)
- [BIPs and EIPs Standards support](#bips-and-eips-standards-support)
- [Security Considerations](#security-considerations)
- [Design Rationale](#design-rationale)
- [P2P Overlay](#p2p-overlay-1)
- [Why devp2p? Why not use libp2p?](#why-devp2p-why-not-use-libp2p)
- [What about other RLPx subprotocols like LES, and Swarm?](#what-about-other-rlpx-subprotocols-like-les-and-swarm)
- [Why do you use Whisper?](#why-do-you-use-whisper)
- [I heard you were moving away from Whisper?](#i-heard-you-were-moving-away-from-whisper)
- [Why is PoW for Whisper set so low?](#why-is-pow-for-whisper-set-so-low)
- [Why do you not use Discovery v5 for node discovery?](#why-do-you-not-use-discovery-v5-for-node-discovery)
- [I heard something about mailservers being trusted somehow?](#i-heard-something-about-mailservers-being-trusted-somehow)
- [Data sync](#data-sync-1)
- [Why is MVDS not used for public chats?](#why-is-mvds-not-used-for-public-chats)
- [Footnotes](#footnotes)
- [Appendix A: Security considerations](#appendix-a-security-considerations)
- [Scalability and UX](#scalability-and-ux)
- [Privacy](#privacy)
- [Spam resistance](#spam-resistance)
- [Censorship resistance](#censorship-resistance)
- [Acknowledgements](#acknowledgements)
### Protocol layers
Implementing a Status clients means implementing the following layers. Additionally, there are separate specifications for things like key management and account lifecycle.
Implementing a Status clients largely means implementing the following layers. Additionally, there are separate specifications for things like key management and account lifecycle.
| Layer | Purpose | Technology |
|-------------------|---------------------------------|------------------------------|
| Data and payloads | End user functionality | 1:1, group chat, public chat |
| Data sync | Data consistency | MVDS Ratchet |
| Secure transport | Confidentiality, PFS, etc | Double Ratchet |
| Transport privacy | Routing, Metadata protection | Whisper |
| P2P Overlay | Overlay routing, NAT traversal | devp2p |
Other aspects, such as how IPFS is used for stickers, how we interact with the Ethereum blockchain or how the browser works, are currently underspecified. These sets of specifications should allow you to implement a a Status client for basic private communication.
| Layer | Purpose | Technology |
| ----------------- | ------------------------------ | ---------------------------- |
| Data and payloads | End user functionality | 1:1, group chat, public chat |
| Data sync | Data consistency | MVDS Ratchet |
| Secure transport | Confidentiality, PFS, etc | Double Ratchet |
| Transport privacy | Routing, Metadata protection | Whisper |
| P2P Overlay | Overlay routing, NAT traversal | devp2p |
### Protobuf
We use [`protobuf`](https://developers.google.com/protocol-buffers/) in different layers, the version used is `proto3` unless stated otherwise.
## Components
### P2P Overlay
Status clients run on the public Ethereum network, as specified by the devP2P
Status clients run on a public, permissionless peer-to-peer network, as specified by the devP2P
network protocols. devP2P provides a protocol for node discovery which is in
draft mode
[here](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md). See
more on node discovery and management in the next section.
To communicate between Ethereum nodes, the [RLPx Transport
To communicate between Status nodes, the [RLPx Transport
Protocol, v5](https://github.com/ethereum/devp2p/blob/master/rlpx.md) is used, which
allows for TCP-based communication between nodes.
On top of this we run the RLPx-based subprotocol [Whisper
v6](https://eips.ethereum.org/EIPS/eip-627) for privacy-preserving messaging.
There MUST be an Ethereum node that is capable of discovering peers and
There MUST be a node that is capable of discovering peers and
implements Whisper V6 specification.
#### Node discovery and roles
@ -95,7 +108,7 @@ There are four types of node roles:
3. Mailservers (servers and clients)
4. Mobile nodes (Status Clients)
To implement a standard Status client you MUST implement the last node type. The
To implement a standard Status client you MUST implement both 2. and 4. node types. The
other node types are optional, but we RECOMMEND you implement a mailserver
client mode, otherwise the user experience is likely to be poor.
@ -107,23 +120,29 @@ nodes allow you to discover other nodes of the network.
Currently the main bootstrap nodes are provided by Status Gmbh, but anyone can
run these provided they are connected to the rest of the Whisper network.
Status maintains a list of boootstrap nodes in the following locations:
- Asia:
- `enode://e8a7c03b58911e98bbd66accb2a55d57683f35b23bf9dfca89e5e244eb5cc3f25018b4112db507faca34fb69ffb44b362f79eda97a669a8df29c72e654416784@47.91.224.35:443`
- `enode://43947863cfa5aad1178f482ac35a8ebb9116cded1c23f7f9af1a47badfc1ee7f0dd9ec0543417cc347225a6e47e46c6873f647559e43434596c54e17a4d3a1e4@47.52.74.140:443`
- Europe:
- `enode://436cc6f674928fdc9a9f7990f2944002b685d1c37f025c1be425185b5b1f0900feaf1ccc2a6130268f9901be4a7d252f37302c8335a2c1a62736e9232691cc3a@174.138.105.243:443`
- `enode://5395aab7833f1ecb671b59bf0521cf20224fe8162fc3d2675de4ee4d5636a75ec32d13268fc184df8d1ddfa803943906882da62a4df42d4fccf6d17808156a87@206.189.243.57:443`
- North America:
- `enode://7427dfe38bd4cf7c58bb96417806fab25782ec3e6046a8053370022cbaa281536e8d64ecd1b02e1f8f72768e295d06258ba43d88304db068e6f2417ae8bcb9a6@104.154.88.123:443`
- `enode://ebefab39b69bbbe64d8cd86be765b3be356d8c4b24660f65d493143a0c44f38c85a257300178f7845592a1b0332811542e9a58281c835babdd7535babb64efc1@35.202.99.224:443`
Status maintains a list of production fleet boootstrap nodes in the following locations:
These bootstrap nodes do not change, however, we can't guarantee that it will stay this way forever
**Hong Kong:**
- `enode://6e6554fb3034b211398fcd0f0082cbb6bd13619e1a7e76ba66e1809aaa0c5f1ac53c9ae79cf2fd4a7bacb10d12010899b370c75fed19b991d9c0cdd02891abad@47.75.99.169:443`
- `enode://23d0740b11919358625d79d4cac7d50a34d79e9c69e16831c5c70573757a1f5d7d884510bc595d7ee4da3c1508adf87bbc9e9260d804ef03f8c1e37f2fb2fc69@47.52.106.107:443`
**Amsterdam:**
- `enode://436cc6f674928fdc9a9f7990f2944002b685d1c37f025c1be425185b5b1f0900feaf1ccc2a6130268f9901be4a7d252f37302c8335a2c1a62736e9232691cc3a@178.128.138.128:443`
- `enode://5395aab7833f1ecb671b59bf0521cf20224fe8162fc3d2675de4ee4d5636a75ec32d13268fc184df8d1ddfa803943906882da62a4df42d4fccf6d17808156a87@178.128.140.188:443`
**Central US:**
- `enode://32ff6d88760b0947a3dee54ceff4d8d7f0b4c023c6dad34568615fcae89e26cc2753f28f12485a4116c977be937a72665116596265aa0736b53d46b27446296a@34.70.75.208:443`
- `enode://5405c509df683c962e7c9470b251bb679dd6978f82d5b469f1f6c64d11d50fbd5dd9f7801c6ad51f3b20a5f6c7ffe248cc9ab223f8bcbaeaf14bb1c0ef295fd0@35.223.215.156:443`
These bootstrap nodes MAY change and we can't guarantee that it will stay this way forever
and at some point we might be forced to change them.
#### Discovery
To implement a Status client you need to discover peers to connect to. We use a
To implement a Status client you MUST discover or have a list of peers to connect to. We use a
light discovery mechanism based on a combination of [Discovery
v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) and
[Rendezvous Protocol](https://github.com/libp2p/specs/tree/master/rendezvous),
@ -149,14 +168,14 @@ Status nodes that want to be discovered MUST register to Discovery V5 and/or Ren
with the `whisper` topic. Status nodes that are mail servers and want to
be discoverable MUST additionally register with the `whispermail` topic.
The recommended strategy is to use both mechanisms but at the same time implement a structure
It is RECOMMENDED to use both mechanisms but at the same time implement a structure
called `PeerPool`. `PeerPool` is responsible for maintaining an optimal number of peers.
For mobile nodes, there is no significant advantage to have more than 2-3 peers and one mail server.
`PeerPool` can notify peers discovery protocol implementations that they should suspend
their execution because the optimal number of peers is found. They should resume
if the number of connected peers drops or a mail server disconnects.
It is worth noticing that an efficient caching strategy can be of great use, especially,
It is worth noticing that an efficient caching strategy MAY be of great use, especially,
on mobile devices. Discovered peers can be cached as they rarely change and used
when the client starts again. In such a case, there might be no need to even start
peers discovery protocols because cached peers will satisfy the optimal number of peers.
@ -166,7 +185,7 @@ way because there is no peers discovery algorithm overhead introduced. The disad
is that these peers might be gone and without peers discovery mechanism, it won't be possible to find
new ones.
The current list of static peers is published on https://fleets.status.im/. `eth.beta` is the current
The current list of static peers is published on <https://fleets.status.im/>. `eth.prod` is the current
group of peers the official Status client uses. The others are test networks.
Finally, Waku node addresses can be retrieved by traversing
@ -183,17 +202,15 @@ communicate with other Status nodes.
Once a Whisper node is up and running there are some specific settings required
to commmunicate with other Status nodes.
See [Status Whisper Usage Spec](status-whisper-usage-spec.md) for more details.
See [3/WHISPER-USAGE](https://specs.status.im/spec/3) for more details.
For providing offline inboxing, see the complementary [Whisper Mailserver
Spec](status-whisper-mailserver-spec.md).
For providing offline inboxing, see the complementary [4/WHISPER-MAILSERVER](https://specs.status.im/spec/4).
### Secure Transport
In order to provide confidentiality, integrity, authentication and forward
secrecy of messages we implement a secure transport on top of Whisper. This is
used in 1:1 chats and group chats, but not for public chats. See [Status Secure
Transport Spec](status-secure-transport-spec.md) for more.
used in 1:1 chats and group chats, but not for public chats. See [5/SECURE-TRANSPORT](https://specs.status.im/spec/5) for more.
### Data Sync
@ -205,30 +222,16 @@ Transport Spec](status-secure-transport-spec.md) for more.
On top of secure transport, we have various types of data sync clients and
payload formats for things like 1:1 chat, group chat and public chat. These have
various degrees of standardization. Please refer to [Initial Message Payload
Specification](status-payloads-spec.md) for more details.
various degrees of standardization. Please refer to [6/PAYLOADS](https://specs.status.im/spec/6) for more details.
### BIPs and EIPs Standards support
For a list of EIPs and BIPs that SHOULD be supported by Status client, please
see [Status EIPs Standards](status-EIPs.md).
see [8/EIPS](https://specs.status.im/spec/8).
## Security Considerations
TBD.
<!-- TODO: Fill this out. -->
### Censorship-resistance
With default settings Whisper over DevP2P runs on odd ports in 30k range, which
are easy to block. One workaround for this is to run ports on 443. This doesn't
take care of all cases though, and this quickly leads into efforts such as
obfuscated transports a la Tor.
See https://github.com/status-im/status-react/issues/6351 for some discussion.
<!-- TODO: More detail on interop of ports and what we do precisely -->
See [Appendix A](#appendix-a-security-considerations)
## Design Rationale
@ -271,10 +274,10 @@ computer.
Whisper is not currently under active development, and it has several drawbacks.
Among others:
- It is very wasteful bandwidth-wise and it doesn't appear to be scalable
- Proof of work is a poor spam protection mechanism for heterogenerous devices
- The privacy guarantees provided are not rigorous
- There's no incentives to run a node
- It is very wasteful bandwidth-wise and it doesn't appear to be scalable
- Proof of work is a poor spam protection mechanism for heterogenerous devices
- The privacy guarantees provided are not rigorous
- There's no incentives to run a node
Finding a more suitable transport privacy is an ongoing research effort,
together with [Vac](https://vac.dev/vac-overview) and other teams in the space.
@ -316,9 +319,9 @@ very bandwidth heavy.
## Footnotes
1. <https://github.com/status-im/status-protocol-go/>
2. <https://github.com/status-im/status-console-client/>
3. <https://github.com/status-im/status-react/>
1. <https://github.com/status-im/status-protocol-go/>
2. <https://github.com/status-im/status-console-client/>
3. <https://github.com/status-im/status-react/>
## Appendix A: Security considerations
@ -380,4 +383,8 @@ A mailserver has a direct TCP connection, which means they are trusted to send t
By default Devp2p runs on port `30303`, which is not commonly used for any other service. This means it is easy to censor, e.g. airport WiFi. This can be mitigated somewhat by running on e.g. port `80` or `443`, but there are still outstanding issues. See libp2p and Tor's Pluggable Transport for how this can be improved.
See <https://github.com/status-im/status-react/issues/6351> for some discussion.
## Acknowledgements
Jacek Sieka

53
status-account-spec.md → docs/stable/2-account.md
View File

@ -1,12 +1,20 @@
# Status Account Specification
---
permalink: /spec/2
parent: Stable specs
title: 2/ACCOUNT
---
> Version: 0.1 (Draft)
# 2/ACCOUNT
> Version: 0.2
>
> Status: Stable
>
> Authors: Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
## Abstract
TBD.
In this specification we explain what Status account is, and how trust is established.
## Table of Contents
@ -16,12 +24,10 @@ TBD.
- [Initial Key Generation](#initial-key-generation)
- [Public/Private Keypairs](#publicprivate-keypairs)
- [X3DH Prekey bundle creation](#x3dh-prekey-bundle-creation)
- [Register at push notification system](#register-at-push-notification-system)
- [Account Broadcasting](#account-broadcasting)
- [X3DH Prekey bundles](#x3dh-prekey-bundles)
- [Optional Account additions](#optional-account-additions)
- [ENS Username](#ens-username)
- [User Chosen Name](#user-chosen-name)
- [User Profile Picture](#user-profile-picture)
- [Trust establishment](#trust-establishment)
- [Terms Glossary](#terms-glossary)
@ -58,14 +64,13 @@ Everything else associated with the contact is either verified or derived from t
- The default paths are defined as such:
- Whisper Chat Key (`IK`): `m/43'/60'/1581'/0'/0` (post Multiaccount integration)
- following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
- DB encryption Key (`DBK`): `m/43'/60'/1581'/1'/0` (post Multiaccount integration)
- following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md)
- Status Wallet paths: `m/44'/60'/0'/0'/i` starting at `i=0`
<!-- WE CURRENTLY DO NOT IMPLEMENT ENCRYPTION KEY, FOR FUTURE - C.P. -->
<!-- - DB encryption Key (`DBK`): `m/43'/60'/1581'/1'/0` (post Multiaccount integration)
- following [EIP1581](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1581.md) -->
- Status Wallet paths: `m/44'/60'/0'/0/i` starting at `i=0`
- following [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki)
- NOTE: this (`i=0`) is also the current (and only) path for Whisper key before Multiaccount integration
<!-- TODO: Remove time dependency, only write what is the case now - i.e. remove "post Multiaccount integration" -->
### X3DH Prekey bundle creation
- Status follows the X3DH prekey bundle scheme that Open Whisper Systems outlines [in their documentation](https://signal.org/docs/specifications/x3dh/#the-x3dh-protocol) with the following exceptions:
- Because there are no central servers, we do not publish one-time keys `OPK` or perform DH including them.
@ -76,14 +81,6 @@ Everything else associated with the contact is either verified or derived from t
- Timestamp
- These bundles are made available in a variety of ways, as defined in section 2.1.
### Register at push notification system
If you want to receive and send push notifications, you MUST register a push
notification server. This part is currently underspecified. You MAY choose to
not do this.
<!-- TODO: Add details on this this. -->
## Account Broadcasting
- A user is responsible for broadcasting certain information publicly so that others may contact them.
@ -94,14 +91,12 @@ not do this.
- TODO: retreival of long-time offline users bundle via `{IK}-contact-code`
## Optional Account additions
### ENS Username
- A user MAY register a public username on the Ethereum Name System (ENS). This username is a user-chosen subdomain of the `stateofus.eth` ENS registration that maps to their whisper identity key (`IK`).
### User Chosen Name
- An account MAY create a display name to replace the `IK` generated 3-word pseudonym in chat screens. This chosen display name will become part of the publicly broadcasted profile of the account.
### User Profile Picture
- An account MAY edit the `IK` generated identicon with a chosen picture. This picture will become part of the publicly broadcasted profile of the account.
<!-- ### User Profile Picture
- An account MAY edit the `IK` generated identicon with a chosen picture. This picture will become part of the publicly broadcasted profile of the account. -->
<!-- TODO: Elaborate on wallet account and multiaccount -->
<!-- TODO: Elaborate on security implications -->
@ -111,18 +106,20 @@ not do this.
**Trust establishment deals with users verifying they are communicating with who they think they are.**
### Terms Glossary
| term | description |
| ---- | ----------- |
| privkey | ECDSA secp256k1 private key |
| pubkey | ECDSA secp256k1 public key |
| whisper key | pubkey for chat with HD derivation path m/44'/60'/0'/0/0 |
| whisper key | pubkey for chat with HD derivation path m/43'/60'/1581'/0'/0 |
### Contact Discovery
#### Public channels
- Public group channels in Status are a broadcast/subscription system. All public messages are encrypted with a symmetric key drived from the channel name, `K_{pub,sym}`, which is publicly known.
- A public group channel's symmetric key MUST creation must follow the [web3 API](https://web3js.readthedocs.io/en/1.0/web3-shh.html#generatesymkeyfrompassword)'s `web3.ssh.generateSymKeyFromPassword` function
- In order to post to a public group channel, a client MUST have a valid account created (as per section [Account Creation Specification](./status-account-spec)).
- In order to post to a public group channel, a client MUST have a valid account created.
- In order to listen to a public group channel, a client must subscribe to the channel name. The sender of a message is derived from the message's signature.
- Discovery of channel names is not currently part of the protocol, and is typically done out of band. If a channel name is used that has not been used, it will be created.
- A client MUST sign the message otherwise it will be discarded by the recipients.
@ -159,12 +156,16 @@ This can be done in the following ways:
- a bundle SHOULD be distributed on the contact code channel. This is the whisper topic `{IK}-contact-code`, where `IK` is the hex encoded public key of the user, prefixed with `0x`. The channel is encrypted in the same way public chats are encrypted.
### Contact Verification
Once you have the information of a contact, the following can be used to verify that the key material is as it should be.
#### Identicon
A low-poly identicon is deterministically generated from the whisper chat public key. This can then be compared out of band to ensure the reciever's public key is the one you have locally.
#### 3 word pseudonym / whisper key fingerprint
Status generates a deterministic 3-word random pseudonym from the whisper chat public key. This pseudonym acts as a human readable fingerprint to the whisper chat public key. This name also shows when viewing a contact's public profile and in the chat UI.
- implementation: [gfycat](https://github.com/status-im/status-react/tree/develop/src/status_im/utils/gfycat)
#### ENS name
Status offers the ability to register a mapping of a human readable subdomain of `stateofus.eth` to their whisper chat public key. This registration is purchased (currently by staking 10 SNT) and stored on the Ethereum mainnet blockchain for public lookup.
@ -217,4 +218,4 @@ All messages sent are encrypted with the public key of the destination and signe
## Security Considerations
TBD.
-

84
status-whisper-usage-spec.md → docs/stable/3-whisper-usage.md
View File

@ -1,6 +1,14 @@
# Status Whisper Usage Specification
---
permalink: /spec/3
parent: Stable specs
title: 3/WHISPER-USAGE
---
> Version: 0.1 (Draft)
# 3/WHISPER-USAGE
> Version: 0.2
>
> Status: Stable
>
> Authors: Adam Babik <adam@status.im>, Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
@ -62,13 +70,13 @@ encryption properties to support asynchronous chat.
| Messages | 1 | ✔ | [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md) |
| PoW Requirement | 2 | ✔ | [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md) |
| Bloom Filter | 3 | ✔ | [EIP-627](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-627.md) |
| Batch Ack | 11 | 𝘅 | TODO |
| Message Response | 12 | 𝘅 | TODO |
| P2P Sync Request | 123 | 𝘅 | TODO |
| P2P Sync Response | 124 | 𝘅 | TODO |
| P2P Request Complete | 125 | 𝘅 | [Status Whisper Mailserver Spec](status-whisper-mailserver-spec.md) |
| P2P Request | 126 | ✔ | [Status Whisper Mailserver Spec](status-whisper-mailserver-spec.md) |
| P2P Messages | 127 | ✔/𝘅 (EIP-627 supports only single envelope in a packet) | [Status Whisper Mailserver Spec](status-whisper-mailserver-spec.md) |
| Batch Ack | 11 | 𝘅 | Undocumented |
| Message Response | 12 | 𝘅 | Undocumented |
| P2P Sync Request | 123 | 𝘅 | Undocumented |
| P2P Sync Response | 124 | 𝘅 | Undocumented |
| P2P Request Complete | 125 | 𝘅 | [4/WHISPER-MAILSERVER](https://specs.status.im/spec/4) |
| P2P Request | 126 | ✔ | [4/WHISPER-MAILSERVER](https://specs.status.im/spec/4) |
| P2P Messages | 127 | ✔/𝘅 (EIP-627 supports only single envelope in a packet) | [4/WHISPER-MAILSERVER](https://specs.status.im/spec/4) |
## Whisper node configuration
@ -126,7 +134,7 @@ The protocol requires a key (symmetric or asymmetric) for the following actions:
As asymmetric keys and symmetric keys are required to process incoming messages,
they must be available all the time and are stored in memory.
Keys management for PFS is described in [Perfect forward secrecy section](#perfect-forward-secrecy-pfs).
Keys management for PFS is described in [5/SECURE-TRANSPORT](https://specs.status.im/spec/5).
The Status protocols uses a few particular Whisper topics to achieve its goals.
@ -155,7 +163,7 @@ for i = 0; i < topicLen; i++ {
### Partitioned topic
Whisper is broadcast-based protocol. In theory, everyone could communicate using a single topic but that would be extremaly inefficient. Opposite would be using a unique topic for each conversation, however, this brings privacy concerns because it would be much easier to detect whether and when two parties have an active conversation.
Whisper is broadcast-based protocol. In theory, everyone could communicate using a single topic but that would be extremely inefficient. Opposite would be using a unique topic for each conversation, however, this brings privacy concerns because it would be much easier to detect whether and when two parties have an active conversation.
Partitioned topics are used to broadcast private messages efficiently. By selecting a number of topic, it is possible to balance efficiency and privacy.
@ -179,8 +187,6 @@ for i = 0; i < topicLen; i++ {
}
```
If partitioned topic support is enabled by the Status client, it MUST listen to its paritioned topic. It MUST be generated using the algorithm above and active public key.
### Public chats
A public chat MUST use a topic derived from a public chat name following the algorithm below:
@ -230,14 +236,50 @@ Generic discovery topic is a legacy topic used to handle all one-to-one chats. T
Generic discovery topic MUST be created following [Public chats](#public-chats) topic algorithm using string `contact-discovery` as a name. -->
### One-to-one topic
In order to receive one-to-one messages incoming from a public key `P`, the Status Client MUST listen to a [Contact Code Topic](#contact-code-topic) created for that public key.
### Group chat topic
Group chats does not have a dedicated topic. All group chat messages (including membership updates) are sent as one-to-one messages to multiple recipients.
### Negotiated topic
When a client sends a one to one message to another client, it MUST listen to their negotiated topic. This is computed by generating
a diffie-hellman key exchange between two members and taking the first four bytes of the `SHA3-256` of the key generated.
```golang
sharedKey, err := ecies.ImportECDSA(myPrivateKey).GenerateShared(
ecies.ImportECDSAPublic(theirPublicKey),
16,
16,
)
hexEncodedKey := hex.EncodeToString(sharedKey)
var hash []byte = keccak256(hexEncodedKey)
var topicLen int = 4
if len(hash) < topicLen {
topicLen = len(hash)
}
var topic [4]byte
for i = 0; i < topicLen; i++ {
topic[i] = hash[i]
}
```
A client SHOULD send to the negotiated topic only if it has received a message from all the devices included in the conversation.
### Flow
To exchange messages with client B, a client A SHOULD:
- Listen to client's B Contact Code Topic to retrieve their bundle information, including a list of active devices
- Send a message on client's B partitioned topic
- Listen to the Negotiated Topic between A & B
- Once a message is received from B, the Negotiated Topic SHOULD be used
## Message encryption
Even though, the protocol specifies an encryption layer that encrypts messages before passing them to the transport layer, Whisper protocol requires each Whisper message to be encrypted anyway.
@ -248,9 +290,9 @@ One-to-one messages are encrypted using asymmetric encryption.
## Message confirmations
Sending a message is a complex process where many things can go wrong. Message confirmations tell a node that a message originating from it has been received by its peers.
Sending a message is a complex process where many things can go wrong. Message confirmations tell a node that a message originating from it has been seen by its direct peers.
A node MAY send a message confirmation for any batch of messages received with a packet Messages Code (`0x01`).
A node MAY send a message confirmation for any batch of messages received in a packet Messages Code (`0x01`).
A message confirmation is sent using Batch Acknowledge packet (`0x0b`) or Message Response packet (`0x0c`).
@ -267,7 +309,9 @@ The Message Response packet is more complex and is followed by a Versioned Messa
The supported codes:
`1`: means time sync error which happens when an envelope is too old or created in the future (the root cause is no time sync between nodes).
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message, a corresponding confirmation is broadcasted by one or more peers.
The drawback of sending message confirmations is that it increases the noise in the network because for each sent message, a corresponding confirmation is broadcasted by one or more peers. To limit that, both Batch Acknowledge packet (`0x0b`) and Message Response packet (`0x0c`) are not broadcasted to peers of the peers, i.e. they do not follow epidemic spread.
In the current Status network setup, only Mailservers support message confirmations. A client posting a message to the network and after receiving a confirmation can be sure that the message got processed by the Mailserver. If additionally, sending a message is limited to non-Mailserver peers, it also guarantees that the message got broadcasted through the network and it reached the selected Mailserver.
## Whisper V6 extensions

20
status-whisper-mailserver-spec.md → docs/stable/4-whisper-mailserver.md
View File

@ -1,5 +1,14 @@
# Status Whisper Mailserver Specification
> Version: 0.1 (Draft)
---
permalink: /spec/4
parent: Stable specs
title: 4/WHISPER-MAILSERVER
---
# 4/WHISPER-MAILSERVER
> Version: 0.2
>
> Status: Stable
>
> Authors: Adam Babik <adam@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
@ -85,7 +94,7 @@ In order to be useful, a mailserver SHOULD be online most of time. That means
you either have to be a bit tech-savvy to run your own node, or rely on someone
else to run it for you.
Currently Status Gmbh provides mailservers in an altruistic manner, but this is
Currently one of Status's legal entities provides mailservers in an altruistic manner, but this is
suboptimal from a decentralization, continuance and risk point of view. Coming
up with a better system for this is ongoing research.
@ -96,8 +105,9 @@ A Status client SHOULD allow the mailserver selection to be customizable.
In order to use a Mailserver, a given node needs to connect to it directly,
i.e. add the Mailserver as its peer and mark it as trusted. This means that the
Mailserver is able to send direct p2p messages to the node instead of
broadcasting them. Effectively, it knows which topics the node is interested in,
when it is online as well as many metadata like IP address.
broadcasting them. Effectively, it will have access to the bloom filter of
topics that the user is interested in, when it is online as well as many
metadata like IP address.
### Denial-of-service

95
status-secure-transport-spec.md → docs/stable/5-secure-transport.md
View File

@ -1,8 +1,16 @@
# Status Secure Transport Specification
---
permalink: /spec/5
parent: Stable specs
title: 5/SECURE-TRANSPORT
---
> Version: 0.1 (Draft)
# 5/SECURE-TRANSPORT
> Version: 0.2
>
> Authors: Andrea Piana <andreap@status.im>, Pedro Pombeiro <pedro@status.im>, Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im>, Dean Eigenmann <dean@status.im
> Status: Stable
>
> Authors: Andrea Piana <andreap@status.im>, Pedro Pombeiro <pedro@status.im>, Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im>, Dean Eigenmann <dean@status.im>
## Abstract
@ -30,6 +38,22 @@ It builds on the [X3DH](https://signal.org/docs/specifications/x3dh/) and [Doubl
- [Initial key exchange flow (X3DH)](#initial-key-exchange-flow-x3dh)
- [Double Ratchet](#double-ratchet)
- [Security Considerations](#security-considerations)
- [Session management](#session-management)
- [Abstract](#abstract)
- [Introduction](#introduction)
- [Initialization](#initialization)
- [Concurrent sessions](#concurrent-sessions)
- [Re-keying](#re-keying)
- [Multi-device support](#multi-device-support)
- [Pairing](#pairing)
- [Sending messages to a paired group](#sending-messages-to-a-paired-group)
- [Account recovery](#account-recovery)
- [Partitioned devices](#partitioned-devices)
- [Trust establishment](#trust-establishment)
- [Contact request](#contact-request)
- [Expired session](#expired-session)
- [Stale devices](#stale-devices)
## Introduction
@ -106,13 +130,14 @@ Every client initially generates some key material which is stored locally:
- A signed prekey based on secp256k1 - `SPK`
- A prekey signature - `Sig(IK, Encode(SPK))`
More details can be found in the `X3DH Prekey bundle creation` section of [Account specification](./status-account-spec.md#x3dh-prekey-bundle-creation).
More details can be found in the `X3DH Prekey bundle creation` section of [2/ACCOUNT](https://specs.status.im/spec/2#x3dh-prekey-bundles).
Prekey bundles can be extracted from any user's messages, or found via searching for their specific topic, `{IK}-contact-code`.
TODO: See below on bundle retrieval, this seems like enhancement and parameter for recommendation
### Bundle retrieval
<!-- TODO: Potentially move this completely over to [Trust Establishment](./status-account-spec.md) -->
X3DH works by having client apps create and make available a bundle of prekeys (the X3DH bundle) that can later be requested by other interlocutors when they wish to start a conversation with a given user.
@ -137,7 +162,7 @@ There are two phases in the initial negotiation of a 1:1 chat:
1. **Identity verification** (e.g., face-to-face contact exchange through QR code, Identicon matching). A QR code serves two purposes simultaneously - identity verification and initial bundle retrieval;
1. **Asynchronous initial key exchange**, using X3DH.
For more information on account generation and trust establishment, see [Status Account Specification](status-account-spec.md)
For more information on account generation and trust establishment, see [2/ACCOUNT](https://specs.status.im/spec/2)
#### Initial key exchange flow (X3DH)
@ -456,3 +481,63 @@ TODO: this requires more detail
- Mailservers act to provide asynchronicity so users can retrieve messages after coming back from an offline period.
-->
## Session management
A peer is identified by two pieces of data:
1) An `installation-id` which is generated upon creating a new account in the `Status` application
2) Their identity whisper key
### Initialization
A new session is initialized once a successful X3DH exchange has taken place. Subsequent messages will use the established session until re-keying is necessary.
### Concurrent sessions
If two sessions are created concurrently between two peers the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
### Re-keying
On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
### Multi-device support
Multi-device support is quite challenging as we don't have a central place where information on which and how many devices (identified by their respective `installation-id`) belongs to a whisper-identity.
Furthermore we always need to take account recovery in consideration, where the whole device is wiped clean and all the information about any previous sessions is lost.
Taking these considerations into account, the way multi-device information is propagated through the network is through x3dh bundles, which will contain information about paired devices as well as information about the sending device.
This mean that every time a new device is paired, the bundle needs to be updated and propagated with the new information, and the burden is put on the user to make sure the pairing is successful.
The method is loosely based on https://signal.org/docs/specifications/sesame/ .
### Pairing
When a user adds a new account in the `Status` application, a new `installation-id` will be generated. The device should be paired as soon as possible if other devices are present. Once paired the contacts will be notified of the new device and it will be included in further communications.
Any time a bundle from your `IK` but different `installation-id` is received, the device will be shown to the user and will have to be manually approved, to a maximum of 3. Once that is done any message sent by one device will also be sent to any other enabled device.
Once a new device is enabled, a new bundle will be generated which will include pairing information.
The bundle will be propagated to contacts through the usual channels.
Removal of paired devices is a manual step that needs to be applied on each device, and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
### Sending messages to a paired group
When sending a message, the peer will send a message to other `installation-id` that they have seen.
The number of devices is capped to 3, ordered by last activity.
Messages are sent using pairwise encryption, including their own devices.
### Account recovery
Account recovery is no different from adding a new device, and it is handled in exactly the same way.
### Partitioned devices
In some cases (i.e. account recovery when no other pairing device is available, device not paired), it is possible that a device will receive a message that is not targeted to its own `installation-id`.
In this case an empty message containing bundle information is sent back, which will notify the receiving end of including this device in any further communication.

335
docs/stable/6-payloads.md Normal file
View File

@ -0,0 +1,335 @@
---
permalink: /spec/6
parent: Stable specs
title: 6/PAYLOADS
---
# 6/PAYLOADS
> Version: 0.2
>
> Status: Stable
>
> Authors: Adam Babik <adam@status.im>, Andrea Maria Piana <andreap@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
## Abstract
This specifications decribes how the payload of each message in Status looks
like. It is primarly centered around chat and chat-related use cases.
The payloads aims be flexible enough to support messaging but also cases
described in the [Status Whitepaper](https://status.im/whitepaper.pdf) as well
as various clients created using different technologies.
## Table of Contents
- [Status Message Payloads Specification](#status-message-payloads-specification)
- [Abstract](#abstract)
- [Table of Contents](#table-of-contents)
- [Introduction](#introduction)
- [Payload wrapper](#payload-wrapper)
- [Encoding](#encoding)
- [Types of messages](#types-of-messages)
- [Message](#message)
- [Payload](#payload)
- [Payload](#payload-1)
- [Content types](#content-types)
- [Sticker content type](#sticker-content-type)
- [Message types](#message-types)
- [Clock vs Timestamp and message ordering](#clock-vs-timestamp-and-message-ordering)
- [Chats](#chats)
- [Contact Update](#contact-update)
- [Payload](#payload-2)
- [Contact update](#contact-update-1)
- [SyncInstallationContact](#syncinstallationcontact)
- [Payload](#payload-3)
- [SyncInstallationPublicChat](#syncinstallationpublicchat)
- [Payload](#payload-4)
- [PairInstallation](#pairinstallation)
- [Payload](#payload-5)
- [MembershipUpdateMessage and MembershipUpdateEvent](#membershipupdatemessage-and-membershipupdateevent)
- [Upgradability](#upgradability)
- [Security Considerations](#security-considerations)
- [Design rationale](#design-rationale)
## Introduction
In this document we describe the payload format and some special considerations.
## Payload wrapper
All payloads are wrapped in a [protobuf record](https://developers.google.com/protocol-buffers/)
record:
```protobuf
message StatusProtocolMessage {
bytes signature = 4001;
bytes payload = 4002;
}
```
`signature` is the bytes of the signed `SHA3-256` of the payload, signed with the key of the author of the message.
The signature is needed to validate authorship of the message, so that the message can be relayed to third parties.
If a signature is not present but an author is provided by a layer below, the message is not to be relayed to third parties and it is considered plausibly deniable.
## Encoding
The payload is encoded using [Protobuf](https://developers.google.com/protocol-buffers)
## Types of messages
### Message
The type `ChatMessage` represents a chat message exchanged between clients.
#### Payload
The protobuf description is:
```protobuf
message ChatMessage {
// Lamport timestamp of the chat message
uint64 clock = 1;
// Unix timestamps in milliseconds, currently not used as we use whisper as more reliable, but here
// so that we don't rely on it
uint64 timestamp = 2;
// Text of the message
string text = 3;
// Id of the message that we are replying to
string response_to = 4;
// Ens name of the sender
string ens_name = 5;
// Chat id, this field is symmetric for public-chats and private group chats,
// but asymmetric in case of one-to-ones, as the sender will use the chat-id
// of the received, while the receiver will use the chat-id of the sender.
// Probably should be the concatenation of sender-pk & receiver-pk in alphabetical order
string chat_id = 6;
// The type of message (public/one-to-one/private-group-chat)
MessageType message_type = 7;
// The type of the content of the message
ContentType content_type = 8;
oneof payload {
StickerMessage sticker = 9;
}
enum MessageType {
UNKNOWN_MESSAGE_TYPE = 0;
ONE_TO_ONE = 1;
PUBLIC_GROUP = 2;
PRIVATE_GROUP = 3;
// Only local
SYSTEM_MESSAGE_PRIVATE_GROUP = 4;}
enum ContentType {
UNKNOWN_CONTENT_TYPE = 0;
TEXT_PLAIN = 1;
STICKER = 2;
STATUS = 3;
EMOJI = 4;
TRANSACTION_COMMAND = 5;
// Only local
SYSTEM_MESSAGE_CONTENT_PRIVATE_GROUP = 6;
}
}
```
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | clock | `uint64` | The clock of the chat|
| 2 | timestamp | `uint64` | The sender timestamp at message creation |
| 3 | text | `string` | The content of the message |
| 4 | response_to | `string` | The ID of the message replied to |
| 5 | ens_name | `string` | The ENS name of the user sending the message |
| 6 | chat_id | `string` | The local ID of the chat the message is sent to |
| 7 | message_type | `MessageType` | The type of message, different for one-to-one, public or group chats |
| 8 | content_type | `ContentType` | The type of the content of the message |
| 9 | payload | `Sticker|nil` | The payload of the message based on the content type |
#### Content types
Content types are required for a proper interpretation of incoming messages. Not each message is plain text but may carry a different information.
The following content types MUST be supported:
* `TEXT_PLAIN` identifies a message which content is a plaintext.
There are also other content types that MAY be implemented by the client:
* `STICKER`
* `STATUS`
* `EMOJI`
* `TRANSACTION_COMMAND`
##### Sticker content type
A `ChatMessage` with `STICKER` `Content/Type` MUST also specify the ID of the `Pack` and
the `Hash` of the pack, in the `Sticker` field of `ChatMessage`
```protobuf
message StickerMessage {
string hash = 1;
int32 pack = 2;
}
```
#### Message types
Message types are required to decide how a particular message is encrypted and what metadata needs to be attached when passing a message to the transport layer. For more on this, see [3/WHISPER-USAGE](https://specs.status.im/spec/3).
<!-- TODO: This reference is a bit odd, considering the layer payloads should interact with is Secure Transport, and not Whisper. This requires more detail -->
The following messages types MUST be supported:
* `ONE_TO_ONE` is a message to the public group
* `PUBLIC_GROUP` is a private message
* `PRIVATE_GROUP` is a message to the private group.
#### Clock vs Timestamp and message ordering
If a user sends a new message before the messages sent while the user was offline are received, the new
message is supposed to be displayed last in a chat. This is where the basic algorithm of Lamport timestamp would fall short
as it's only meant to order causally related events.
The status client therefore makes a "bid", speculating that it will beat the current chat-timestamp, s.t. the status client's
Lamport timestamp format is: `clock = `max({timestamp}, chat_clock + 1)`
This will satisfy the Lamport requirement, namely: a -> b then T(a) < T(b)
`timestamp` MUST be Unix time calculated when the message is created in milliseconds. This field SHOULD not be relied upon for message ordering.
`clock` SHOULD be calculated using the algorithm of [Lamport timestamps](https://en.wikipedia.org/wiki/Lamport_timestamps). When there are messages available in a chat, `clock`'s value is calculated based on the last received message in a particular chat: `max(timeNowInMs, last-message-clock-value + 1)`. If there are no messages, `clock` is initialized with `timestamp`'s value.
Messages with a `clock` greater than `120` seconds over the whisper timestamp SHOULD be discarded, in order to avoid malicious users to increase the `clock` of a chat arbitrarily.
Messages with a `clock` less than `120` seconds under the whisper timestamp might indicate an attempt to insert messages in the chat history which is not distinguishable from a `datasync` layer re-transit event. A client MAY mark this messages with a warning to the user, or discard them.
`clock` value is used for the message ordering. Due to the used algorithm and distributed nature of the system, we achieve casual ordering which might produce counterintuitive results in some edge cases. For example, when one joins a public chat and sends a message before receiving the exist messages, their message `clock` value might be lower and the message will end up in the past when the historical messages are fetched.
#### Chats
Chat is a structure that helps organize messages. It's usually desired to display messages only from a single recipient or a group of recipients at a time and chats help to achieve that.
All incoming messages can be matched against a chat. Below you can find a table that describes how to calculate a chat ID for each message type.
|Message Type|Chat ID Calculation|Direction|Comment|
|------------|-------------------|---------|-------|
|PUBLIC_GROUP|chat ID is equal to a public channel name; it should equal `chatId` from the message|Incoming/Outgoing||
|ONE_TO_ONE|let `P` be a public key of the recipient; `hex-encode(P)` is a chat ID; use it as `chatId` value in the message|Outgoing||
|user-message|let `P` be a public key of message's signature; `hex-encode(P)` is a chat ID; discard `chat-id` from message|Incoming|if there is no matched chat, it might be the first message from public key `P`; you can discard it or create a new chat; Status official clients create a new chat|
|PRIVATE_GROUP|use `chatId` from the message|Incoming/Outgoing|find an existing chat by `chatId`; if none is found, we are not a member of that chat or we haven't joined that chat, the message MUST be discarded |
### Contact Update
`ContactUpdate` is a message exchange to notify peers that either the
user has been added as a contact, or that information about the sending user have
changed.
```protobuf
message ContactUpdate {
uint64 clock = 1;
string ens_name = 2;
string profile_image = 3;
}
```
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | clock | `uint64` | The clock of the chat with the user |
| 2 | ens_name | `string` | The ENS name if set |
| 3 | profile_image | `string` | The base64 encoded profile picture of the user |
#### Contact update
A client SHOULD send a `ContactUpdate` to all the contacts each time:
- The ens_name has changed
- The profile image is edited
A client SHOULD also periodically send a `ContactUpdate` to all the contacts, the interval is up to the client, the Status official client sends these updates every 48 hours.
### SyncInstallationContact
`SyncInstallationContact` messages are used to synchronize in a best-effort the contacts to other devices.
```protobuf
message SyncInstallationContact {
uint64 clock = 1;
string id = 2;
string profile_image = 3;
string ens_name = 4;
uint64 last_updated = 5;
repeated string system_tags = 6;
}
```
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | clock | `uint64` | clock value of the chat |
| 2 | id | `string` | id of the contact synced |
| 3 | profile_image | `string` | `base64` encoded profile picture of the user |
| 4 | ens_name | `string` | ENS name of the contact |
| 5 | `array[string]` | Array of `system_tags` for the user, this can currently be: `":contact/added", ":contact/blocked", ":contact/request-received"`|
### SyncInstallationPublicChat
`SyncInstallationPublicChat` message is used to synchronize in a best-effort the public chats to other devices.
```protobuf
message SyncInstallationPublicChat {
uint64 clock = 1;
string id = 2;
}
```
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | clock | `uint64` | clock value of the chat |
| 2 | id | `string` | id of the chat synced |
### PairInstallation
`PairInstallation` messages are used to propagate informations about a device to its paired devices.
```protobuf
message PairInstallation {
uint64 clock = 1;
string installation_id = 2;
string device_type = 3;
string name = 4;
}
```
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | clock | `uint64` | clock value of the chat |
| 2| installation_id | `string` | A randomly generated id that identifies this device |
| 3 | device_type | `string` | The OS of the device `ios`,`android` or `desktop` |
| 4 | name | `string` | The self-assigned name of the device |
### MembershipUpdateMessage and MembershipUpdateEvent
`MembershipUpdateEvent` is a message used to propagate information about group membership changes in a group chat.
The details are in the [Group chats specs](status-group-chats-spec.md)
## Upgradability
There are two ways to upgrade the protocol without breaking compatibility:
- Accretion is always supported
- Deletion of existing fields or messages is not supported and might break compatibility
## Security Considerations
-
## Design rationale

12
status-EIPs.md → docs/stable/8-eips.md
View File

@ -1,6 +1,14 @@
# Status EIPs standards
---
permalink: /spec/8
parent: Stable specs
title: 8/EIPS
---
> Version: 0.1 (Draft)
# 8/EIPS
> Version: 0.2
>
> Status: Stable
>
> Authors: Ricardo Guilherme Schmidt <ricardo3@status.im>

9
docs/stable/stable.md Normal file
View File

@ -0,0 +1,9 @@
---
layout: default
title: Stable specs
nav_order: 1
has_children: true
permalink: /specs/stable
---
# Stable specifications

5
home.html Normal file
View File

@ -0,0 +1,5 @@
---
layout: default
---
{{ content }}

16
package.json Normal file
View File

@ -0,0 +1,16 @@
{
"name": "specs",
"dependencies": {
"remark-cli": "^6.0.1",
"remark-lint": "^6.0.2",
"remark-preset-lint-recommended": "^3.0.2"
},
"scripts": {
"lint": "remark ."
},
"remarkConfig": {
"plugins": [
"remark-preset-lint-recommended"
]
}
}

5
post.html Normal file
View File

@ -0,0 +1,5 @@
---
layout: default
---
{{ content }}

204
status-group-chats-spec.md
View File

@ -1,204 +0,0 @@
# Status Group Chat Specification
> Version: 0.1 (Draft)
>
> Authors: Andrea Maria Piana <andreap@status.im>
>
## Table of Contents
- [Abstract](#abstract)
- [Membership updates](#membership-updates)
- [Chat ID](#chat-id)
- [Signature](#signature)
- [Group membership event](#group-membership-event)
- [chat-created](#chat-created)
- [name-changed](#name-changed)
- [members-added](#members-added)
- [members-joined](#members-joined)
- [admins-added](#admins-added)
- [members-removed](#members-removed)
- [admin-removed](#admin-removed)
## Abstract
This documents describes the group chat protocol used by the status application. Pairwise encryption is used among member so a message is exchanged between each participants, similarly to a one-to-one message.
## Membership updates
Membership updates messages are used to propagate group chat membership changes. The transit format is described in the [Status Payload Specs](status-payload-specs.md). Here we will be describing each specific field.
The format is:
```
{
"events": [struct {"type": string, "member": string, "members": [string], "clock-value": uint, "name": string],
"signature": string,
"chat-id": string
}
```
### Chat ID
Each membership update MUST be sent with a corresponding `chat-id`.
The format of this chat id MUST be a string, [UUID](https://tools.ietf.org/html/rfc4122 ), concatenated with the hex-encoded public key of the creator of the chat. This chat-id MUST be validated by all clients, and MUST be discarded if it does not follow these rules.
### Signature
The signature for each event is calculated by creating a JSON array of all the `events` sorted by `clock-value` in ascending order, where each event is transformed in an array of tuples `field-name`, `value`, sorted by `field-name` in ascending alphabetical order. The last element of the array MUST be the `chat-id`.
Empty fields MUST be removed.
For example the event:
```
{
"chat-id": "chat-id",
"events": [
{"b": "b-value"
"clock-value": 1,
"a": "a-value"
},
{
"e": "e-value",
"clock-value": 0,
"a": "a-value"
}
]
}
```
Results in the structure:
```
[
[
[
["a" "a-value"],
["clock-value", 0],
["e" "e-value"]
],
[
["a", "a-value"],
["b", "b-value"],
["clock-value", 1]
]
],
"chat-id"
]
```
This structure is then stringified collapsing all whitespaces and the `Keccak256` of the string is then signed using its private key by the author and added to the payload.
### Group membership event
Any group membership event received MUST be verified by calculating the signature as per the method described above.
The author MUST be extracted from it, if the verification fails the event MUST be discarded.
#### chat-created
```
{
"type": "chat-created",
"name": string
"clock-value": uint
}
```
Chat created event is the first event that needs to be sent. Any event with a clock value lower then this MUST be discarded.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and create a chat with identified by `chat-id` and named `name`.
#### name-changed
```
{
"type": "name-changed"
"name": string
"clock-value": uint
}
```
A name changed event is used by admins to change the name of the group chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
If the event is valid the chat name SHOULD be changed to `name`.
#### members-added
```
{
"type": "members-added"
"members": [string]
"clock-value": uint
}
```
A members added event is used by admins to add members to the chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
If the event is valid a client MUST update the list of members of the chat who have not joined, adding the `members` received.
`members` is an array of hex encoded public keys.
#### member-joined
```
{
"type": "member-joined"
"member": string
"clock-value": uint
}
```
A members joined event is used by a member of the chat to signal that they want to start receiving messages from this chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and MUST ensure the author of the event is the same as the one specified by the `member` field.
If the event is valid a client MUST update the list of members of the chat who joined, adding `member`. Any `message` sent to the group chat should now include the newly joined member.
#### admins-added
```
{
"type": "admins-added"
"members": [string]
"clock-value": uint
}
```
An admins added event is used by admins to add make other admins in the chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates, MUST ensure the author of the event is an admin of the chat and MUST ensure all `members` are already `members` of the chat, otherwise the event MUST be ignored.
If the event is valid a client MUST update the list of admins of the chat, adding the `members` received.
`members` is an array of hex encoded public keys.
#### member-removed
```
{
"type": "member-removed"
"member": string
"clock-value": uint
}
```
A member-removed event is used to leave or kick members of the chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates, MUST ensure that:
- If the author of the event is an admin, target can only be themselves or a non-admin member.
- If the author of the event is not an admin, the target of the event can only be themselves.
-
If the event is valid a client MUST remove the member from the list of `members`/`admins` of the chat, and no further message should be sent to them.
#### admin-removed
```
{
"type": "admin-removed"
"member": string
"clock-value": uint
}
```
An admin-removed event is used to drop admin privileges.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates, MUST ensure that the author of the event is also the target of the event.
If the event is valid a client MUST remove the member from the list of `admins` of the chat.

227
status-payloads-spec.md
View File

@ -1,227 +0,0 @@
# Status Message Payloads Specification
> Version: 0.1 (Draft)
>
> Authors: Adam Babik <adam@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
## Abstract
This specifications decribes how the payload of each message in Status looks
like. It is primarly centered around chat and chat-related use cases.
The payloads aims be flexible enough to support messaging but also cases
described in the [Status Whitepaper](https://status.im/whitepaper.pdf) as well
as various clients created using different technologies.
## Table of Contents
- [Status Message Payloads Specification](#status-message-payloads-specification)
- [Abstract](#abstract)
- [Table of Contents](#table-of-contents)
- [Introduction](#introduction)
- [Payload wrapper](#payload-wrapper)
- [Encoding](#encoding)
- [Types of Messages] (#types-of-messages)
- [Message](#message)
- [Payload](#payload)
- [Content types](#content-types)
- [Message types](#message-types)
- [Clock vs Timestamp and message ordering](#clock-vs-timestamp-and-message-ordering)
- [Chats](#chats)
- [Contact requests](#contact-requests)
- [Payload] (#payload)
- [Contact update] (#contact-update)
- [Handling contact messages] (#handling-contact-messages)
- [SyncInstallation](#sync-installation)
- [Payload](#payload)
- [PairInstallation](#pair-installation)
- [Payload](#payload)
- [GroupMembershipUpdate](#group-membership-update)
- [Payload](#payload)
- [Upgradability](#upgradability)
- [Security Considerations](#security-considerations)
- [Design rationale](#design-rationale)
- [Why are you using Transit and Protobuf?](#why-are-you-using-transit-and-protobuf)
## Introduction
In this document we describe the payload format and some special considerations.
## Payload wrapper
All payloads are wrapped in a [protobuf record](https://developers.google.com/protocol-buffers/)
record:
```protobuf
message StatusProtocolMessage {
bytes signature = 4001;
bytes payload = 4002;
}
```
`signature` is the bytes of the signed `SHA3-256` of the payload, signed with the key of the author of the message.
The signature is needed to validate authorship of the message, so that the message can be relayed to third parties.
If a signature is not present but an author is provided by a layer below, the message is not to be relayed to third parties and its considered plausibly deniable.
## Encoding
The payload is encoded using [Transit format](https://github.com/cognitect/transit-format). Transit was chosen over JSON in order to reduce the bandwidth.
## Types of messages
### Message
The type `Message` represents a text message exchanged between clients and is identified by the transit tag `c4`.
#### Payload
Payload is a struct (a compound data type) with the following fields (order is important):
<!-- TODO: Be more precise in struct description, a la RFC, e.g. TLS style https://tools.ietf.org/html/rfc8446 -->
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | text | `string` | The text version of the message content |
| 2 | content type | `enum` (more in [Content types](#content-types)) | See details |
| 3 | message type | `enum` (more in [Message types](#message-types)) | See details |
| 4 | clock | `int64` | See details |
| 5 | timestamp | `int64` | See details |
| 6 | content | `struct { chat-id string, text string, response-to string }` | The chat-id of the chat this message is destined to, the text of the content and optionally the id of the message it is responding to|
#### Content types
Content types are required for a proper interpretation of incoming messages. Not each message is plain text but may carry a different information.
The following content types MUST be supported:
* `text/plain` identifies a message which content is a plain text.
There are also other content types that MAY be implemented by the client:
* `sticker`
* `status`
* `command`
* `command-request`
* `emoji`
These are currently underspecified. We refer to real-world implementations for clients who wish to interoperate.
<!-- TODO: Ideally specify this, but barring that, link to implementation. -->
#### Message types
Message types are required to decide how a particular message is encrypted and what metadata needs to be attached when passing a message to the transport layer. For more on this, see [Status Whisper Usage Specification](./status-whisper-usage-spec.md).
<!-- TODO: This reference is a bit odd, considering the layer payloads should interact with is Secure Transport, and not Whisper. This requires more detail -->
The following messages types MUST be supported:
* `public-group-user-message` is a message to the public group
* `user-message` is a private message
* `group-user-message` is a message to the private group.
#### Clock vs Timestamp and message ordering
`timestamp` MUST be Unix time calculated when the message is created in milliseconds. This field SHOULD not be relied upon for message ordering.
`clock` SHOULD be calculated using the algorithm of [Lamport timestamps](https://en.wikipedia.org/wiki/Lamport_timestamps). When there are messages available in a chat, `clock`'s value is calculated based on the last received message in a particular chat: `last-message-clock-value + 1`. If there are no messages, `clock` is initialized with `timestamp * 100`'s value.
`clock` value is used for the message ordering. Due to the used algorithm and distributed nature of the system, we achieve casual ordering which might produce counterintuitive results in some edge cases. For example, when one joins a public chat and sends a message before receiving the exist messages, their message `clock` value might be lower and the message will end up in the past when the historical messages are fetched.
#### Chats
Chat is a structure that helps organize messages. It's usually desired to display messages only from a single recipient or a group of recipients at a time and chats help to achieve that.
All incoming messages can be matched against a chat. Below you can find a table that describes how to calculate a chat ID for each message type.
|Message Type|Chat ID Calculation|Direction|Comment|
|------------|-------------------|---------|-------|
|public-group-user-message|chat ID is equal to a public channel name; it should equal `chat-id` from message's `content` field|Incoming/Outgoing||
|user-message|let `P` be a public key of the recipient; `hex-encode(P)` is a chat ID; use it as `chat-id` value in message's `content` field|Outgoing||
|user-message|let `P` be a public key of message's signature; `hex-encode(P)` is a chat ID; discard `chat-id` from message's `content` field|Incoming|if there is no matched chat, it might be the first message from public key `P`; you can discard it or create a new chat; Status official clients create a new chat|
|group-user-message|use `chat-id` from message's `content` field|Incoming/Outgoing|find an existing chat by `chat-id`; if none is found discard the message (TODO: incomplete)|
<!-- TODO: "group-user-message" is not complete. Does it require to explicitly join the group chat? Is there a way to invite someone? Also, if I start a new group chat (or join an existing one), I need to somehow calculate this chatID by myself. How to do it? -->
### Contact Requests
Contact requests consists in 3 kind of messages: `ContactRequest`, `ContactRequestConfirmed` and `ContactUpdate`.
These messages are used to notify the receiving end that it has been added to the sender's contact. They are identified by the transit tags `c2`, `c3`, `c4` respectively, but they are all interchangeable, meaning a client SHOULD handle them in exactly the same way. The payload of the 3 messages is identical.
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | name | `string` | The self-assigned name of the user (DEPRECATED) |
| 2 | profile image | `string` | The base64 encoded profile picture of the user |
| 3 | address | `string` | The ethereum address of the user |
| 4 | fcm-token | `string` | The FCM Token used by mobile devices for push notifications (DEPRECATED) |
| 5 | device-info | `[struct { id string, fcm-token string }]` | A list of pair `installation-id`, `fcm-token` for each device that is currently paired |
#### Contact update
A client SHOULD send a `ContactUpdate` to all the contacts each time:
- The name is edited
- The profile image is edited
- A new device has been paired
A client SHOULD also periodically send a `ContactUpdate` to all the contacts, the interval is up to the client, the Status official client sends these updates every 48 hours.
#### Handling contact messages
A client SHOULD handle any `Contact*` message in the same way. Any `Contact*` message with a whisper timestamp lower than the last one processed MUST be discarded.
### SyncInstallation
`SyncInstallation` messages are used to synchronize in a best-effort way all the paired installations. It is identified by a transit tag of `p1`
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1| contacts | `[struct { name string last-updated int device-info struct {id string fcm-token string } pending? bool}` | An array of contacts |
| 2 | account | `struct {name string photo-path string last-updated int}` | Information about your own account |
| 3 | chat | `struct {:public? bool :chat-id string}` | A description of a public chat opened by the client |
### PairInstallation
`PairInstallation` messages are used to propagate informations about a device to its paired devices. It is identified by a transit tag of `p2`
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1| installation-id | `string` | A randomly generated id that identifies this device |
| 2 | device-type | `string` | The OS of the device `ios`,`android` or `desktop` |
| 3 | name | `string` | The self-assigned name of the device |
| 4 | fcm-token | `string` | The FCM Token used by mobile devices for push notifications |
### GroupMembershipUpdate
`GroupMembershipUpdate` is a message used to propagate information about group membership changes in a group chat.. It is identified by a transit tag of `g5`.
The details are in the [Group chats specs](status-group-chats-spec.md)
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1| chat-id | `string` | The chat id of the chat where the change is to take place |
| 2 | membership-updates | See details | A list of events that describe the membership changes |
| 3 | message | `Transit message` | An optional message, described in [Message](#message) |
## Upgradability
There are two ways to upgrade the protocol without breaking compatibility:
- Struct fields can be enriched with a new key, which will be ignored by old clients.
- An element can be appended to the `Transit` array, which will also be ignored by old clients.
## Security Considerations
TBD.
## Design rationale
### Why are you using Transit and Protobuf?
Transit was initially chose for encoding, and Protobuf was added afterwards. This is partly due to the history of the protocol living inside of `status-react`, which is written in Clojurescript.

93
status-session-management-spec.md
View File

@ -1,93 +0,0 @@
# Status Session Management Specification
> Version: 0.1 (Draft)
>
> Authors: Dean Eigenmann <dean@status.im>, Andrea Piana <andreap@status.im>, Pedro Pombeiro <pedro@status.im>, Corey Petty <corey@status.im>, Oskar Thorén <oskar@status.im>
## Abstract
In this specification we describe how status sessions are handled.
<!-- TODO: Clarify what we mean by a session -->
## Table of Contents
- [Abstract](#abstract)
- [Introduction](#introduction)
- [Initialization](#initialization)
- [Concurrent sessions](#concurrent-sessions)
- [Re-keying](#re-keying)
- [Multi-device support](#multi-device-support)
- [Pairing](#pairing)
- [Sending messages to a paired group](#sending-messages-to-a-paired-group)
- [Account recovery](#account-recovery)
- [Partitioned devices](#partitioned-devices)
- [Trust establishment](#trust-establishment)
- [Contact request](#contact-request)
- [Expired session](#expired-session)
- [Stale devices](#stale-devices)
## Introduction
A peer is identified by two pieces of data:
1) An `installation-id` which is generated upon creating a new account in the `Status` application
2) Their identity whisper key
## Initialization
A new session is initialized once a successful X3DH exchange has taken place. Subsequent messages will use the established session until re-keying is necessary.
## Concurrent sessions
If two sessions are created concurrently between two peers the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
## Re-keying
On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
## Multi-device support
Multi-device support is quite challenging as we don't have a central place where information on which and how many devices (identified by their respective `installation-id`) belongs to a whisper-identity.
Furthermore we always need to take account recovery in consideration, where the whole device is wiped clean and all the information about any previous sessions is lost.
Taking these considerations into account, the way multi-device information is propagated through the network is through x3dh bundles, which will contain information about paired devices as well as information about the sending device.
This mean that every time a new device is paired, the bundle needs to be updated and propagated with the new information, and the burden is put on the user to make sure the pairing is successful.
The method is loosely based on https://signal.org/docs/specifications/sesame/ .
<!-- TODO: This multi device section isn't clear enough -->
<!-- TODO: Additionally, it seems tightly coupled with secure transport, which makes things like multi device public chats harder to reason about (IMO). E.g. as a client impl I might want multi device support but not want to impl double ratchet etc, so what does this mean? -->
<!-- It is coupled to the secure transport because otherwise there's no need of multidevice. Without a secure transport multi-device is trivial (nothing to implement, such in public chats, nothing to reason about), the type of secure transport we use dictates the type of multi-device support we want, same as signal's "Sesame was designed for use with Double Ratchet sessions created via X3DH key agreement.". Please read the specs of sesame, it clearly shows that it's tightly coupled to the encryption layer and its purpose is to allow encrypting messages for multiple devices, such in our case. Let's take some time understanding and reading things before commenting. -->
## Pairing
When a user adds a new account in the `Status` application, a new `installation-id` will be generated. The device should be paired as soon as possible if other devices are present. Once paired the contacts will be notified of the new device and it will be included in further communications.
Any time a bundle from your `IK` but different `installation-id` is received, the device will be shown to the user and will have to be manually approved, to a maximum of 3. Once that is done any message sent by one device will also be sent to any other enabled device.
Once a new device is enabled, a new bundle will be generated which will include pairing information.
The bundle will be propagated to contacts through the usual channels.
Removal of paired devices is a manual step that needs to be applied on each device, and consist simply in disabling the device, at which point pairing information will not be propagated anymore.
## Sending messages to a paired group
When sending a message, the peer will send a message to any `installation-id` that they have seen, using pairwise encryption, including their own devices.
The number of devices is capped to 3, ordered by last activity.
## Account recovery
Account recovery is no different from adding a new device, and it is handled in exactly the same way.
## Partitioned devices
In some cases (i.e. account recovery when no other pairing device is available, device not paired), it is possible that a device will receive a message that is not targeted to its own `installation-id`.
In this case an empty message containing bundle information is sent back, which will notify the receiving end of including this device in any further communication.
## Trust establishment
Trust establishment deals with users verifying they are communicating with who they think they are.