status-im
/
specs
mirror of https://github.com/status-im/specs.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into cleanup-whisper-usage

This commit is contained in:
Adam Babik 2019-10-08 09:28:56 +02:00
parent 5da3f1cf2a 9a9c373f14
commit 7443a0674c
No known key found for this signature in database
GPG Key ID: ED02515A1FC0D1B4
8 changed files with 379 additions and 137 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -20,7 +20,7 @@ No accepted SIPs right now.
The following SIPs are under consideration for standardization.
- [Status Client Specification](status-client-spec.md). The main specification for writing a Status client.
- [Status Client Specification](status-client-spec.md). The main specification for writing a Status client. **Start here**
- [Status Secure Transport Specification](status-secure-transport-spec.md). How Status provide a secure transport with conversational security properties.
- [Status Payload Specification](status-payloads-spec.md). What the message payloads look like.
- [Status Account Specification](status-account-spec.md). What a Status account is and how trust is established.

11
status-account-spec.md
View File

@ -89,8 +89,8 @@ not do this.
### X3DH Prekey bundles
- A client SHOULD regenerate a new X3DH prekey bundle every 24 hours. This MAY be done in a lazy way, such that a client that does not come online past this time period does not regenerate or broadcast bundles.
- The current bundle MUST be broadcast on a whisper topic specific to his Identity Key, `{IK}-contact-code`, intermittently. This MAY be done every 6 hours.
- A bundle MUST accompany every message sent.
- The current bundle SHOULD be broadcast on a whisper topic specific to his Identity Key, `{IK}-contact-code`, intermittently. This MAY be done every 6 hours.
- A bundle SHOULD accompany every message sent.
- TODO: retreival of long-time offline users bundle via `{IK}-contact-code`
## Optional Account additions
@ -131,7 +131,7 @@ not do this.
- is not a public key
#### Private 1:1 messages
This can be done in a the following ways:
This can be done in the following ways:
1. scanning a user generated QR code
1. discovery through the Status app
1. asyncronous X3DH key exchange
@ -156,10 +156,7 @@ This can be done in a the following ways:
- include BundleContainer???
- a new bundle SHOULD be created at least every 12 hours
- a bundle is only generated when it is used
- a bundle MUST be distributed on the contact code channel (NOTE: define this where?)
#### QR code
- A generated QR code should include a X3DH bundle set along with the contact code but I can't find the code to do so.
- a bundle SHOULD be distributed on the contact code channel. This is the whisper topic `{IK}-contact-code`, where `IK` is the hex encoded public key of the user, prefixed with `0x`. The channel is encrypted in the same way public chats are encrypted.
### Contact Verification
Once you have the information of a contact, the following can be used to verify that the key material is as it should be.

204
status-group-chats-spec.md Normal file
View File

@ -0,0 +1,204 @@
# Status Group Chat Specification
> Version: 0.1 (Draft)
>
> Authors: Andrea Maria Piana <andreap@status.im>
>
## Table of Contents
- [Abstract](#abstract)
- [Membership updates](#membership-updates)
- [Chat ID](#chat-id)
- [Signature](#signature)
- [Group membership event](#group-membership-event)
- [chat-created](#chat-created)
- [name-changed](#name-changed)
- [members-added](#members-added)
- [members-joined](#members-joined)
- [admins-added](#admins-added)
- [members-removed](#members-removed)
- [admin-removed](#admin-removed)
## Abstract
This documents describes the group chat protocol used by the status application. Pairwise encryption is used among member so a message is exchanged between each participants, similarly to a one-to-one message.
## Membership updates
Membership updates messages are used to propagate group chat membership changes. The transit format is described in the [Status Payload Specs](status-payload-specs.md). Here we will be describing each specific field.
The format is:
```
{
"events": [struct {"type": string, "member": string, "members": [string], "clock-value": uint, "name": string],
"signature": string,
"chat-id": string
}
```
### Chat ID
Each membership update MUST be sent with a corresponding `chat-id`.
The format of this chat id MUST be a string, [UUID](https://tools.ietf.org/html/rfc4122 ), concatenated with the hex-encoded public key of the creator of the chat. This chat-id MUST be validated by all clients, and MUST be discarded if it does not follow these rules.
### Signature
The signature for each event is calculated by creating a JSON array of all the `events` sorted by `clock-value` in ascending order, where each event is transformed in an array of tuples `field-name`, `value`, sorted by `field-name` in ascending alphabetical order. The last element of the array MUST be the `chat-id`.
Empty fields MUST be removed.
For example the event:
```
{
"chat-id": "chat-id",
"events": [
{"b": "b-value"
"clock-value": 1,
"a": "a-value"
},
{
"e": "e-value",
"clock-value": 0,
"a": "a-value"
}
]
}
```
Results in the structure:
```
[
[
[
["a" "a-value"],
["clock-value", 0],
["e" "e-value"]
],
[
["a", "a-value"],
["b", "b-value"],
["clock-value", 1]
]
],
"chat-id"
]
```
This structure is then stringified collapsing all whitespaces and the `Keccak256` of the string is then signed using its private key by the author and added to the payload.
### Group membership event
Any group membership event received MUST be verified by calculating the signature as per the method described above.
The author MUST be extracted from it, if the verification fails the event MUST be discarded.
#### chat-created
```
{
"type": "chat-created",
"name": string
"clock-value": uint
}
```
Chat created event is the first event that needs to be sent. Any event with a clock value lower then this MUST be discarded.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and create a chat with identified by `chat-id` and named `name`.
#### name-changed
```
{
"type": "name-changed"
"name": string
"clock-value": uint
}
```
A name changed event is used by admins to change the name of the group chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
If the event is valid the chat name SHOULD be changed to `name`.
#### members-added
```
{
"type": "members-added"
"members": [string]
"clock-value": uint
}
```
A members added event is used by admins to add members to the chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and MUST ensure the author of the event is an admin of the chat, otherwise the event MUST be ignored.
If the event is valid a client MUST update the list of members of the chat who have not joined, adding the `members` received.
`members` is an array of hex encoded public keys.
#### member-joined
```
{
"type": "member-joined"
"member": string
"clock-value": uint
}
```
A members joined event is used by a member of the chat to signal that they want to start receiving messages from this chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates and MUST ensure the author of the event is the same as the one specified by the `member` field.
If the event is valid a client MUST update the list of members of the chat who joined, adding `member`. Any `message` sent to the group chat should now include the newly joined member.
#### admins-added
```
{
"type": "admins-added"
"members": [string]
"clock-value": uint
}
```
An admins added event is used by admins to add make other admins in the chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates, MUST ensure the author of the event is an admin of the chat and MUST ensure all `members` are already `members` of the chat, otherwise the event MUST be ignored.
If the event is valid a client MUST update the list of admins of the chat, adding the `members` received.
`members` is an array of hex encoded public keys.
#### member-removed
```
{
"type": "member-removed"
"member": string
"clock-value": uint
}
```
A member-removed event is used to leave or kick members of the chat.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates, MUST ensure that:
- If the author of the event is an admin, target can only be themselves or a non-admin member.
- If the author of the event is not an admin, the target of the event can only be themselves.
-
If the event is valid a client MUST remove the member from the list of `members`/`admins` of the chat, and no further message should be sent to them.
#### admin-removed
```
{
"type": "admin-removed"
"member": string
"clock-value": uint
}
```
An admin-removed event is used to drop admin privileges.
Upon receiving this event a client MUST validate the `chat-id` provided with the updates, MUST ensure that the author of the event is also the target of the event.
If the event is valid a client MUST remove the member from the list of `admins` of the chat.

143
status-payloads-spec.md
View File

@ -21,12 +21,23 @@ as various clients created using different technologies.
- [Introduction](#introduction)
- [Payload wrapper](#payload-wrapper)
- [Encoding](#encoding)
- [Types of Messages] (#types-of-messages)
- [Message](#message)
- [Payload](#payload)
- [Content types](#content-types)
- [Message types](#message-types)
- [Clock vs Timestamp and message ordering](#clock-vs-timestamp-and-message-ordering)
- [Chats](#chats)
- [Contact requests](#contact-requests)
- [Payload] (#payload)
- [Contact update] (#contact-update)
- [Handling contact messages] (#handling-contact-messages)
- [SyncInstallation](#sync-installation)
- [Payload](#payload)
- [PairInstallation](#pair-installation)
- [Payload](#payload)
- [GroupMembershipUpdate](#group-membership-update)
- [Payload](#payload)
- [Upgradability](#upgradability)
- [Security Considerations](#security-considerations)
- [Design rationale](#design-rationale)
@ -56,45 +67,30 @@ If a signature is not present but an author is provided by a layer below, the me
The payload is encoded using [Transit format](https://github.com/cognitect/transit-format). Transit was chosen over JSON in order to reduce the bandwidth.
Example of a valid encoded payload:
## Types of messages
```
["~#c4",["abc123","text/plain","~:public-group-user-message",154593077368201,1545930773682,["^ ","~:chat-id","testing-adamb","~:text","abc123"]]]
```
### Message
The message is an array and each index value has its meaning:
* 0: `c4` is a decoder handler identification for the current payload format. Identifications allow to register handlers for many different types of payload
* 1: array which items correspond to the described payload fields above
The type `Message` represents a text message exchanged between clients and is identified by the transit tag `c4`.
For more details regarding serialization and deserialization please consult [transit format](https://github.com/cognitect/transit-format) specification.
<!-- TODO: This requires a lot more detail since c4 is only one of several types, and also possibly links to implementation
ANDREA: Not sure this section is really needed (other then a brief mention of the fact that we use transit), explaining how transit is encoded is outside of the scope of this document, as well because that's not the only way transit can be encoded. -->
## Message
The type `Message` represents a text message exchanged between clients.
<!-- TODO: It is not clear how this relates to StatusProtocolMessage above -->
### Payload
#### Payload
Payload is a struct (a compound data type) with the following fields (order is important):
<!-- TODO: Be more precise in struct description, a la RFC, e.g. TLS style https://tools.ietf.org/html/rfc8446 -->
| Field | Name | Type |
| ----- | ---- | ---- |
| 1 | text | `string` |
| 2 | content type | `enum` (more in [Content types](#content-types)) |
| 3 | message type | `enum` (more in [Message types](#message-types)) |
| 4 | clock | `int64` |
| 5 | timestamp | `int64` |
| 6 | content | `struct { chat-id string, text string }` |
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | text | `string` | The text version of the message content |
| 2 | content type | `enum` (more in [Content types](#content-types)) | See details |
| 3 | message type | `enum` (more in [Message types](#message-types)) | See details |
| 4 | clock | `int64` | See details |
| 5 | timestamp | `int64` | See details |
| 6 | content | `struct { chat-id string, text string, response-to string }` | The chat-id of the chat this message is destined to, the text of the content and optionally the id of the message it is responding to|
### Content types
#### Content types
Content types are required for a proper interpretation of incoming messages. Not each message is a plain text but may carry a different information.
Content types are required for a proper interpretation of incoming messages. Not each message is plain text but may carry a different information.
The following content types MUST be supported:
* `text/plain` identifies a message which content is a plain text.
@ -110,7 +106,7 @@ These are currently underspecified. We refer to real-world implementations for c
<!-- TODO: Ideally specify this, but barring that, link to implementation. -->
### Message types
#### Message types
Message types are required to decide how a particular message is encrypted and what metadata needs to be attached when passing a message to the transport layer. For more on this, see [Status Whisper Usage Specification](./status-whisper-usage-spec.md).
@ -122,19 +118,15 @@ The following messages types MUST be supported:
* `user-message` is a private message
* `group-user-message` is a message to the private group.
### Clock vs Timestamp and message ordering
#### Clock vs Timestamp and message ordering
`timestamp` MUST be Unix time calculated when the message is created. Because the peers in the Whisper network should have synchronized time, `timestamp` values should be fairly accurate among all Whisper network participants.
`timestamp` MUST be Unix time calculated when the message is created in milliseconds. This field SHOULD not be relied upon for message ordering.
`clock` SHOULD be calculated using the algorithm of [Lamport timestamps](https://en.wikipedia.org/wiki/Lamport_timestamps). When there are messages available in a chat, `clock`'s value is calculated based on the last received message in a particular chat: `last-message-clock-value + 1`. If there are no messages, `clock` is initialized with `timestamp * 100`'s value.
`clock` value is used for the message ordering. Due to the used algorithm and distributed nature of the system, we achieve casual ordering which might produce counterintuitive results in some edge cases. For example, when one joins a public chat and sends a message before receiving the exist messages, their message `clock` value might be lower and the message will end up in the past when the historical messages are fetched.
<!-- TODO: Document section on replies
TODO: Document timestamp, is it in seconds/ms ? -->
## Chats
<!-- This section should probably fall under Message, as it's only valid for Message-type messages -->
#### Chats
Chat is a structure that helps organize messages. It's usually desired to display messages only from a single recipient or a group of recipients at a time and chats help to achieve that.
@ -149,10 +141,80 @@ All incoming messages can be matched against a chat. Below you can find a table
<!-- TODO: "group-user-message" is not complete. Does it require to explicitly join the group chat? Is there a way to invite someone? Also, if I start a new group chat (or join an existing one), I need to somehow calculate this chatID by myself. How to do it? -->
### Contact Requests
Contact requests consists in 3 kind of messages: `ContactRequest`, `ContactRequestConfirmed` and `ContactUpdate`.
These messages are used to notify the receiving end that it has been added to the sender's contact. They are identified by the transit tags `c2`, `c3`, `c4` respectively, but they are all interchangeable, meaning a client SHOULD handle them in exactly the same way. The payload of the 3 messages is identical.
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1 | name | `string` | The self-assigned name of the user (DEPRECATED) |
| 2 | profile image | `string` | The base64 encoded profile picture of the user |
| 3 | address | `string` | The ethereum address of the user |
| 4 | fcm-token | `string` | The FCM Token used by mobile devices for push notifications (DEPRECATED) |
| 5 | device-info | `[struct { id string, fcm-token string }]` | A list of pair `installation-id`, `fcm-token` for each device that is currently paired |
#### Contact update
A client SHOULD send a `ContactUpdate` to all the contacts each time:
- The name is edited
- The profile image is edited
- A new device has been paired
A client SHOULD also periodically send a `ContactUpdate` to all the contacts, the interval is up to the client, the Status official client sends these updates every 48 hours.
#### Handling contact messages
A client SHOULD handle any `Contact*` message in the same way. Any `Contact*` message with a whisper timestamp lower than the last one processed MUST be discarded.
### SyncInstallation
`SyncInstallation` messages are used to synchronize in a best-effort way all the paired installations. It is identified by a transit tag of `p1`
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1| contacts | `[struct { name string last-updated int device-info struct {id string fcm-token string } pending? bool}` | An array of contacts |
| 2 | account | `struct {name string photo-path string last-updated int}` | Information about your own account |
| 3 | chat | `struct {:public? bool :chat-id string}` | A description of a public chat opened by the client |
### PairInstallation
`PairInstallation` messages are used to propagate informations about a device to its paired devices. It is identified by a transit tag of `p2`
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1| installation-id | `string` | A randomly generated id that identifies this device |
| 2 | device-type | `string` | The OS of the device `ios`,`android` or `desktop` |
| 3 | name | `string` | The self-assigned name of the device |
| 4 | fcm-token | `string` | The FCM Token used by mobile devices for push notifications |
### GroupMembershipUpdate
`GroupMembershipUpdate` is a message used to propagate information about group membership changes in a group chat.. It is identified by a transit tag of `g5`.
The details are in the [Group chats specs](status-group-chats-spec.md)
#### Payload
| Field | Name | Type | Description |
| ----- | ---- | ---- | ---- |
| 1| chat-id | `string` | The chat id of the chat where the change is to take place |
| 2 | membership-updates | See details | A list of events that describe the membership changes |
| 3 | message | `Transit message` | An optional message, described in [Message](#message) |
## Upgradability
The current protocol format is hardly upgradable without breaking backward compatibility. Because Transit is used in this particular way described above, the only reliable option is to append a new field to the Transit record definition. It will be simply ignored by the old clients.
<!-- Not sure I agree with this statement, seems very arbitrary, appending to an array is just as upgradable as adding an entry in a map, just less convenient, I would remove the qualitative statement, and just describe how to upgrade -->
There are two ways to upgrade the protocol without breaking compatibility:
- Struct fields can be enriched with a new key, which will be ignored by old clients.
- An element can be appended to the `Transit` array, which will also be ignored by old clients.
## Security Considerations
@ -162,5 +224,4 @@ TBD.
### Why are you using Transit and Protobuf?
Transit was initially chose for encoding, and Protobuf was added afterwards. This is partly due to the history of the protocol living inside of `status-react`, which is written in Clojurescript. In future versions of payload and data sync client specifications it is likely we'll move towards Protobuf only. See e.g. [Dasy](https://github.com/vacp2p/dasy) for a research proof of concept.
<!-- I would remove the link to dasy, I find it a bit confusing, and the repo just implements something totally different and a fraction of the functionalities -->
Transit was initially chose for encoding, and Protobuf was added afterwards. This is partly due to the history of the protocol living inside of `status-react`, which is written in Clojurescript.

21
status-secure-transport-spec.md
View File

@ -46,9 +46,9 @@ In this document we describe how a secure channel is established, and how variou
- **Confidentiality**: The adversary should not be able to learn what data is being exchanged between two Status clients.
- **Authenticity**: The adversary should not be able to cause either endpoint of a Status 1:1 chat to accept data from any third party as though it came from the other endpoint.
- **Forward Secrecy**: The adversary should not be able to learn what data was exchanged between two Status clients if, at some later time, the adversary compromises one or both of the endpoint devices.
- **Integrity**: The adversary should not be able to cause either endpoint of a Status 1:1 chat to accept data that has been tampered with.
<!-- TODO: Integrity should be here -->
<!-- TODO: It is not clearly stated in this spec how we achieve confidentiality, authenticity and integrity. State this clearly. -->
All of these properties are ensured by the use of [Signal's Double Ratchet](https://signal.org/docs/specifications/doubleratchet/)
### Conventions
@ -84,6 +84,8 @@ Furthermore, Status uses the concept of prekeys (through the use of [X3DH](https
Status uses the following cryptographic primitives:
- Whisper
- AES-256-GCM
- ECIES
- ECDSA
- KECCAK-256
- X3DH
- Elliptic curve Diffie-Hellman key exchange (secp256k1)
@ -106,9 +108,7 @@ Every client initially generates some key material which is stored locally:
More details can be found in the `X3DH Prekey bundle creation` section of [Account specification](./status-account-spec.md#x3dh-prekey-bundle-creation).
A `contact-code` is a protobuf `Bundle` message, encoded in `JSON` and converted to their `base64` string representation.
Prekey bundles are can be extracted from any user's messages, or found via searching for their specific contact code topic, `{IK}-contact-code`.
Prekey bundles can be extracted from any user's messages, or found via searching for their specific topic, `{IK}-contact-code`.
TODO: See below on bundle retrieval, this seems like enhancement and parameter for recommendation
@ -127,6 +127,8 @@ In the X3DH specification, a shared server is typically used to store bundles an
<!-- TODO: Comment, it isn't clear what we actually _do_. It seems as if this is exploring the problem space. From a protocol point of view, it might make sense to describe the interface, and then have a recommendation section later on that specifies what we do. See e.g. Signal's specs where they specify specifics later on. -->
Currently only public and one-to-one message exchanges and Whisper is used to exchange bundles.
Since bundles stored in QR codes or ENS records cannot be updated to delete already used keys, the approach taken is to rotate more frequently the bundle (once every 24 hours), which will be propagated by the app through the channel available.
### 1:1 chat contact request
@ -190,8 +192,6 @@ The initial message sent by Alice to Bob is sent as a top-level `ProtocolMessage
``` protobuf
message ProtocolMessage {
Bundle bundle = 1;
string installation_id = 2;
repeated Bundle bundles = 3;
@ -205,7 +205,6 @@ message ProtocolMessage {
}
```
- `bundle`: optional bundle is exchanged with each message, deprecated;
- `bundles`: a sequence of bundles
- `installation_id`: the installation id of the sender
- `direct_message` is a map of `DirectMessageProtocol` indexed by `installation-id`
@ -285,7 +284,7 @@ TODO: description here
> No honest party will accept a message that has been modified in transit.
- Yes.
- Assuming a user validates (TODO: Check this assumption) every message they are able to decrypt and validates its signature from the sender, then it is not able to be altered in transit.
- Assuming a user validates (TODO: Check this assumption) every message they are able to decrypt and validate its signature from the sender, then it is not able to be altered in transit.
* [igorm] i'm really not sure about it, Whisper provides a signature, but I'm not sure we check it anywhere (simple grepping didn't give anything)
* [andrea] Whisper checks the signature and a public key is derived from it, we check the public key is a meaningful public key. The pk itself is not in the content of the message for public chats/1-to-1 so potentially you could send a message from a random account without having access to the private key, but that would not be much of a deal, as you might just as easily create a random account)
@ -356,7 +355,7 @@ TODO: Verify if this can be done already by looking at Lamport clock difference
#### Message Unlinkability (NO)
> If a judge is convinced that a participant authored one message in the conversation, this does not provide evidence that they authored other messages
- Currently, the Status software signs every messages sent with the user's public key, thus making it no able to give unlinkability.
- Currently, the Status software signs every messages sent with the user's public key, thus making it unable to provide unlinkability.
- This is not necessary though, and could be built in to have an option to not sign.
- Side note: moot account allows for this but is a function of the anonymity set that uses it. The more people that use this account the stronger the unlinkability.
@ -390,7 +389,7 @@ TODO: Verify if this can be done already by looking at Lamport clock difference
- Accept invitation to group
- Leave group
- Non-Members:
- Invited by admins show up as "invited" in group; this leaks contacat information
- Invited by admins show up as "invited" in group; this leaks contact information
- Invited people don't opt-in to being invited
TODO: Group chat dynamics should have a documented state diagram

35
status-session-management-spec.md
View File

@ -39,11 +39,11 @@ A new session is initialized once a successful X3DH exchange has taken place. Su
## Concurrent sessions
If two sessions are created concurrently between two peers the one with the symmetric key, first in byte order should be used this marks that the other has expired.
If two sessions are created concurrently between two peers the one with the symmetric key first in byte order SHOULD be used, this marks that the other has expired.
## Re-keying
On receiving a bundle from a given peer with a higher version, the old bundle should be marked as expired and a new session should be established on the next message sent.
On receiving a bundle from a given peer with a higher version, the old bundle SHOULD be marked as expired and a new session SHOULD be established on the next message sent.
## Multi-device support
@ -51,7 +51,7 @@ Multi-device support is quite challenging as we don't have a central place where
Furthermore we always need to take account recovery in consideration, where the whole device is wiped clean and all the information about any previous sessions is lost.
Taking these considerations into account, the way multi-device information is propagated through the network is through bundles/contact codes, which will contain information about paired devices as well as information about the sending device.
Taking these considerations into account, the way multi-device information is propagated through the network is through x3dh bundles, which will contain information about paired devices as well as information about the sending device.
This mean that every time a new device is paired, the bundle needs to be updated and propagated with the new information, and the burden is put on the user to make sure the pairing is successful.
@ -59,6 +59,7 @@ The method is loosely based on https://signal.org/docs/specifications/sesame/ .
<!-- TODO: This multi device section isn't clear enough -->
<!-- TODO: Additionally, it seems tightly coupled with secure transport, which makes things like multi device public chats harder to reason about (IMO). E.g. as a client impl I might want multi device support but not want to impl double ratchet etc, so what does this mean? -->
<!-- It is coupled to the secure transport because otherwise there's no need of multidevice. Without a secure transport multi-device is trivial (nothing to implement, such in public chats, nothing to reason about), the type of secure transport we use dictates the type of multi-device support we want, same as signal's "Sesame was designed for use with Double Ratchet sessions created via X3DH key agreement.". Please read the specs of sesame, it clearly shows that it's tightly coupled to the encryption layer and its purpose is to allow encrypting messages for multiple devices, such in our case. Let's take some time understanding and reading things before commenting. -->
## Pairing
@ -66,7 +67,7 @@ When a user adds a new account in the `Status` application, a new `installation-
Any time a bundle from your `IK` but different `installation-id` is received, the device will be shown to the user and will have to be manually approved, to a maximum of 3. Once that is done any message sent by one device will also be sent to any other enabled device.
Once a new device is enabled, a new contact-code/bundle will be generated which will include pairing information.
Once a new device is enabled, a new bundle will be generated which will include pairing information.
The bundle will be propagated to contacts through the usual channels.
@ -90,29 +91,3 @@ In this case an empty message containing bundle information is sent back, which
## Trust establishment
Trust establishment deals with users verifying they are communicating with who they think they are.
<!-- TODO: Deduplicate this and status accounts trust establishment -->
### Contact request
Once two accounts have been generated (Alice and Bob), Alice can send a contact request with an introductory message to Bob.
There are two possible scenarios, which dictate the presence or absence of a prekey bundle:
1. If Alice is using Bob's public chat key or ENS name, no prekey bundle is present;
1. If Alice found Bob through the app or scanned Bob's QR code, a prekey bundle is embedded and can be used to set up a secure channel as described in the [Initial key exchange flow X3DH](#initial-key-exchange-flow-X3DH) section.
Bob receives a contact request, informing him of:
- Alice's introductory message.
If Bob's prekey bundle was not available to Alice, Perfect Forward Secrecy hasn't yet been established. In any case, there are no implicit guarantees that Alice is whom she claims to be, and Bob should perform some form of external verification (e.g., using an Identicon).
If Bob accepts the contact request, a secure channel is created (if it wasn't already), and a visual indicator is displayed to signify that PFS has been established. Bob and Alice can then start exchanging messages, making use of the Double Ratchet algorithm as explained in more detail in [Double Ratchet](#double-ratchet) section.
If Bob denies the request, Alice is not able to send messages and the only action available is resending the contact request.
## Expired session
Expired session should not be used for new messages and should be deleted after 14 days from the expiration date, in order to be able to decrypt out-of-order and mailserver messages.
## Stale devices
When a bundle is received from `IK` a timer is initiated on any `installation-id` belonging to `IK` not included in the bundle. If after 7 days no bundles are received from these devices they are marked as `stale` and no message will be sent to them.

63
status-whisper-mailserver-spec.md
View File

@ -3,63 +3,54 @@
>
> Authors: Adam Babik <adam@status.im>, Oskar Thorén <oskar@status.im> (alphabetical order)
- [Status Whisper Mailserver Specification](#status-whisper-mailserver-specification)
- [Abstract](#abstract)
- [Mailserver](#mailserver)
- [Archiving messages](#archiving-messages)
- [Delivering messages](#delivering-messages)
- [Security considerations](#security-considerations)
- [Confidentiality](#confidentiality)
- [Altruistic and centralized operator risk](#altruistic-and-centralized-operator-risk)
- [Privacy concerns](#privacy-concerns)
- [Denial-of-service](#denial-of-service)
## Abstract
Status clients are often offline. In order to allow clients to talk to each other while one is offline, we provide offline inboxing.
Being mostly offline is an intrinsic property of mobile clients. They need to save network transfer and battery consumption to avoid spending too much money or constant charging. Whisper protocol, on the other hand, is an online protocol. Messages are available in the Whisper network only for short period of time calculate in seconds.
This current specification is an extension of Whisper v6 and operates under a store-and-forward model.
## Table of Contents
TBD.
## Introduction
In the case of mobile clients which are often offline, there is a strong need to have an ability to download offline messages. By offline messages, we mean messages sent into the Whisper network and expired before being collected by the recipient. A message stays in the Whisper network for a duration specified as `TTL` (time-to-live) property.
See [EIP-627](https://eips.ethereum.org/EIPS/eip-627) for more detail on *Whisper Mail Server* and *Whisper Mail Client*.
Whisper Mailserver is a Whisper extension that allows to store messages permanently and deliver them to the clients even though they are already not available in the network and expired.
## Mailserver
A mailserver can either be running as a server or as a client.
From the network perspective, Mailserver is just like any other Whisper node. The only different is that it has a capability of archiving messages and delivering them to its peers on-demand.
Since Whisper is a form of DHT, a mailserver only requires a specific relationship with the receiver of a message, not with the sender of a message.
It is important to notice that Mailserver will only handle requests from its direct peers and exchanged packets between Mailserver and a peer are p2p messages.
### Server
### Archiving messages
<!-- TODO: This doesn't actually describe how to implement a mailserver -->
In order to store messages, one MUST implement the interface below and MUST register it within a Whisper service. The only known Whisper implementation that allows that is [geth](https://github.com/ethereum/go-ethereum).
`MailServer` is an interface with two methods:
`MailServer` interface consist of:
* `Archive(env *Envelope)`
* `DeliverMail(whisperPeer *Peer, request *Envelope)`
### Client
### Delivering messages
A Whisper client needs to register a mail server instance which will be used by [geth's Whisper service](https://github.com/ethereum/go-ethereum/blob/v1.8.23/whisper/whisperv6/whisper.go#L209-L213).
Mailserver delivers archieved messages to a peer after receiving a Whisper packet with code `p2pRequestCode`. Messages are delivered asynchronously, i.e. a requester sends a Whisper packet with code `p2pRequestCode` and at some point later, it will start receiving Whisper packets with code `p2pMessageCode`.
If a mail server is registered for a given Whisper client, it will save all incoming messages on a local disk (this is the simplest implementation, it can store the messages wherever it wants, also using technologies like swarm and IPFS) in the background.
How a peer can initialize the request to a Mailserver is up to the implementator. Status peers acting as Mailserver expose two additional JSON-RPC methods: `shhext_requestMessages` and `shh_requestMessagesSync`.
Notice that each node is meant to be independent and SHOULD keep a copy of all historic messages. High Availability (HA) can be achieved by having multiple nodes in different locations. Additionally, each node is free to store messages in a way which provides storage HA as well.
Saved messages are delivered to a requester (another Whisper peer) asynchronously as a response to `p2pMessageCode` message code. This is not exposed as a JSON-RPC method in `shh` namespace but it's exposed in status-go as `shhext_requestMessages` and blocking `shh_requestMessagesSync`. Read more about [Whisper V6 extensions](#whisper-v6-extensions-or-status-whisper-node).
In order to receive historic messages from a filter, p2p messages MUST be allowed when creating the filter. Receiving p2p messages is implemented in [geth's Whisper V6 implementation](https://github.com/ethereum/go-ethereum/blob/v1.8.23/whisper/whisperv6/whisper.go#L739-L751).
Because all packets exchanged between a Mailserver and a peer are p2p packets, all filters created by a peer from which it expectes to receive archived messages MUST allow processing of direct peer-to-peer messages.
## Security considerations
### Confidentiality
All Whisper envelopes are encrypted, and a mailserver node can't inspect their contents.
### High-availability
Since mailservers rely on being online to receive messages on behalf of other clients, this puts a high-availability requirement on individual nodes.
In practice, it is best to treat individual nodes as a form of a cache, and ensure consistency of messages at a different layer. See data sync layer.
All Whisper envelopes are encrypted. Mailserver node can not inspect their contents.
### Altruistic and centralized operator risk
In order to be useful, a mailserver has to be online most of time. That means
In order to be useful, a mailserver SHOULD be online most of time. That means
you either have to be a bit tech-savvy to run your own node, or rely on someone
else to run it for you.
@ -71,9 +62,9 @@ A Status client SHOULD allow the mailserver selection to be customizable.
### Privacy concerns
In order to use a mail server, a given node needs to connect to it directly,
i.e. add the mail server as its peer and mark it as trusted. This means that the
mail server is able to send direct p2p messages to the node instead of
In order to use a Mailserver, a given node needs to connect to it directly,
i.e. add the Mailserver as its peer and mark it as trusted. This means that the
Mailserver is able to send direct p2p messages to the node instead of
broadcasting them. Effectively, it knows which topics the node is interested in,
when it is online as well as many metadata like IP address.

25
status-whisper-usage-spec.md
View File

@ -9,7 +9,7 @@
- [Reason](#reason)
- [Terminology](#terminology)
- [Whisper node configuration](#whisper-node-configuration)
- [Topics](#topics)
- [Keys management](#keys-management)
- [Contact code topic](#contact-code-topic)
- [Partitioned topic](#partitioned-topic)
- [Public chats](#public-chats)
@ -53,11 +53,26 @@ encryption properties to support asynchronous chat.
## Whisper node configuration
Whisper's Proof Of Work algorithm is used to deter denial of service and various spam/flood attacks against the Whisper network. The sender of a message MUST perform some work which in this case means processing time. Because Status' main client is a mobile client, this easily leads to battery draining and poor performance of the app itself. Hence, all clients MUST use the following Whisper node settings:
* proof-of-work not larger than `0.002`
If you want to run a Whisper node and receive messages from Status clients, it must be properly configured.
Whisper's Proof Of Work algorithm is used to deter denial of service and various spam/flood attacks against the Whisper network. The sender of a message must perform some work which in this case means processing time. Because Status' main client is a mobile client, this easily leads to battery draining and poor performance of the app itself. Hence, all clients MUST use the following Whisper node settings:
* proof-of-work requirement not larger than `0.002`
* time-to-live not lower than `10` (in seconds)
## Topics
<!-- TODO: provide an instruction how to start a Whisper node with proper configuration using geth.-->
<!-- @TODO: is there a higher bound -->
## Keys management
The protocol requires a key (symmetric or asymmetric) for the following actions:
* signing & verifying messages (asymmetric key)
* encrypting & decrypting messages (asymmetric or symmetric key).
As asymmetric keys and symmetric keys are required to process incoming messages,
they must be available all the time and are stored in memory.
Keys management for PFS is described in [Perfect forward secrecy section](#perfect-forward-secrecy-pfs).
The Status protocols uses a few particular Whisper topics to achieve its goals.
@ -165,7 +180,7 @@ In order to receive one-to-one messages incoming from a public key `P`, the Stat
### Group chat topic
TODO
Group chats does not have a dedicated topic. All group chat messages (including membership updates) are sent as one-to-one messages to multiple recipients.
## Message encryption