rfc/mvds.md

# Minimum Viable Data Synchronization

> Version: 0.6.0 (Draft)
> 
> Authors: Oskar Thorén <oskar@status.im>, Dean Eigenmann <dean@status.im>

## Table of Contents

1. [Abstract](#abstract)
2. [Definitions](#definitions)
3. [Wire Protocol](#wire-protocol)
    1. [Secure Transport](#secure-transport) 
    2. [Payloads](#payloads)
4. [Synchronization](#synchronization)
    1. [State](#state)
    2. [Flow](#flow)
    3. [Retransmission](#retransmission)
5. [Formal Specification](#formal-Specification)
6. [Footnotes](#footnotes)
7. [Changelog](#changelog)
8. [Acknowledgements](#acknowledgements)
9. [Copyright](#copyright)

## Abstract

In this specification, we describe a minimum viable protocol for data synchronization inspired by the Bramble Synchronization Protocol<sup>1</sup>. This protocol is designed to ensure reliable messaging between peers across an unreliable peer-to-peer (P2P) network where they may be unreachable or unresponsive.

We present a reference implementation<sup>2</sup> including a simulation to demonstrate its performance.

## Definitions

| Term       | Description                                                                         |
|------------|-------------------------------------------------------------------------------------|
| **Peer**   | The other nodes that a node is connected to.                                        |
| **Record** | Defines a payload element of either the type `OFFER`, `REQUEST`, `MESSAGE` or `ACK` |
| **Node**   | Some process that is able to store data, do processing and communicate for MVDS.    |

## Wire Protocol

### Secure Transport

This specification does not define anything related to the transport of packets. It is assumed that this is abstracted in such a way that any secure transport protocol could be easily implemented. Likewise, properties such as confidentiality, integrity, authenticity and forward secrecy are assumed to be provided by a layer below.

### Payloads

Payloads are implemented using [protocol buffers v3](https://developers.google.com/protocol-buffers/).

```protobuf
syntax = "proto3";

package vac.mvds;

message Payload {
  repeated bytes acks = 1;
  repeated bytes offers = 2;
  repeated bytes requests = 3;
  repeated Message messages = 4;
}

message Message {
  bytes group_id = 1;
  int64 timestamp = 2;
  bytes body = 3;
}
```

*The payload field numbers are kept more "unique" to ensure no overlap with other protocol buffers.*

Each payload contains the following fields:

- **Acks:** This field contains a list (can be empty) of `message identifiers` informing the recipient that sender holds a specific message.
- **Offers:** This field contains a list (can be empty) of `message identifiers` that the sender would like to give to the recipient.
- **Requests:** This field contains a list (can be empty) of `message identifiers` that the sender would like to receive from the recipient.
- **Messages:** This field contains a list of messages (can be empty).

**Message Identifiers:** Each `message` has a message identifier calculated by hashing the `group_id`, `timestamp` and `body` fields as follows:

```
HASH("MESSAGE_ID", group_id, timestamp, body);
```

**Group Identifiers:** Each `message` is assigned into a **group** using the `group_id` field, groups are independent synchronization contexts between peers.

The current `HASH` function used is `sha256`.

## Synchronization

### State

We refer to `state` as set of records for the types `OFFER`, `REQUEST` and `MESSAGE` that every node SHOULD store per peer. `state` MUST NOT contain `ACK` records as we do not retransmit those periodically. The following information is stored for records:

 - **Type** - Either `OFFER`, `REQUEST` or `MESSAGE`
 - **Send Count** - The amount of times a record has been sent to a peer.
 - **Send Epoch** - The next epoch at which a record can be sent to a peer.

### Flow

A maximum of one payload SHOULD be sent to peers per epoch, this payload contains all `ACK`, `OFFER`, `REQUEST` and `MESSAGE` records for the specific peer. Payloads are created every epoch, containing reactions to previously received records by peers or new records being sent out by nodes. 

Nodes MAY have two modes with which they can send records: `BATCH` and `INTERACTIVE` mode. The following rules dictate how nodes construct payloads every epoch for any given peer for both modes.

> ***NOTE:** A node may send messages both in interactive and in batch mode.*

#### Interactive Mode

 - A node initially offers a `MESSAGE` when attempting to send it to a peer. This means an `OFFER` is added to the next payload and state for the given peer.
 - When a node receives an `OFFER`, a `REQUEST` is added to the next payload and state for the given peer. 
 - When a node receives a `REQUEST` for a previously sent `OFFER`, the `OFFER` is removed from the state and the corresponding `MESSAGE` is added to the next payload and state for the given peer.
 - When a node receives a `MESSAGE`, the `REQUEST` is removed from the state and an `ACK` is added to the next payload for the given peer.
 - When a node receives an `ACK`, the `MESSAGE` is removed from the state for the given peer.
 - All records that require retransmission are added to the payload, given `Send Epoch` has been reached.

<p align="center">
    <img src="./assets/mvds/interactive.png" />
    <br />
    Figure 1: Delivery without retransmissions in interactive mode.
</p>

#### Batch Mode

 1. When a node sends a `MESSAGE`, it is added to the next payload and the state for the given peer.
 2. When a node receives a `MESSAGE`, an `ACK` is added to the next payload for the corresponding peer.
 3. When a node receives an `ACK`, the `MESSAGE` is removed from the state for the given peer.
 4. All records that require retransmission are added to the payload, given `Send Epoch` has been reached.

<!-- diagram -->

<p align="center">
    <img src="./assets/mvds/batch.png" />
    <br />
    Figure 2: Delivery without retransmissions in batch mode.
</p>


> ***NOTE:** Batch mode is higher bandwidth whereas interactive mode is higer latency.*

<!-- Interactions with state, flow chart with retransmissions? -->

### Retransmission

The record of the type `Type` SHOULD be retransmitted every time `Send Epoch` is smaller than or equal to the current epoch.

`Send Epoch` and `Send Count` MUST be increased every time a record is retransmitted. Although no function is defined on how to increase `Send Epoch`, it SHOULD be exponentially increased until reaching an upper bound where it then goes back to a lower epoch in order to prevent a record's `Send Epoch`'s from becoming too large.

> ***NOTE:** We do not retransmission `ACK`s as we do not know when they have arrived, therefore we simply resend them every time we receive a `MESSAGE`.*

## Formal Specification

MVDS has been formally specified using TLA+: <https://github.com/vacp2p/formalities/tree/master/MVDS>.

## Footnotes

1. <https://code.briarproject.org/briar/briar-spec/blob/master/protocols/BSP.md>
2. <https://github.com/vacp2p/mvds>

## Changelog

| Version | Comment |
| :-----: | ------- |
| [0.6.0](https://github.com/vacp2p/specs/commit/b6335947e09502ec399954002d983a709065a342#diff-a6dbd535bfcb545a64b1c95f6dc8f9f1) | Breaking change to update protocol buffer and field changes. |
| [0.5.2](https://github.com/vacp2p/specs/blob/0adddcdb8d7e71240c2df30d4ba40e49b4275011/mvds.md) | Breaking change to update protocol buffer and field changes. |
| [0.5.1](https://github.com/status-im/bigbrother-specs/blob/20e2659447dcb5123a105526eb26816307c3780c/data_sync/mvds.md) | Documentation updates: clarifications on spec, RFC-2119 and message identifiers. |
| [0.5.0](https://github.com/status-im/bigbrother-specs/blob/6d3d0771b46941791eabe9263f80cbaeaf3d1148/data_sync/mvds.md) | Initial Release |

## Acknowledgements
 - Preston van Loon
 - Greg Markou
 - Rene Nayman
 - Jacek Sieka

## Copyright

Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).