acid-info
/
vac.dev
mirror of https://github.com/acid-info/vac.dev.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

more tweaks

This commit is contained in:
Oskar Thoren 2019-10-02 15:03:29 +08:00
parent 962ca2a760
commit 6a4da5f2b9
No known key found for this signature in database
GPG Key ID: B2ECCFD3BC2EF77E
1 changed files with 3 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
_posts/2019-10-01-remote-log.md
View File

@ -41,8 +41,6 @@ The spec is in an early draft stage and can be found [here](https://github.com/v
| NS | Name system. Associates mutable data to a name. |
| Remote log | Replication of a local log at a different location. |
## Synchronization
### Roles
There are four fundamental roles:
@ -52,7 +50,7 @@ There are four fundamental roles:
2. Name system (NS)
3. Content-addressed storage (CAS)
The *remote log* protobuf is what is stored in the name system.
The *remote log* is the data format of what is stored in the name system.
"Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.
@ -66,7 +64,7 @@ The *remote log* protobuf is what is stored in the name system.
Figure 1: Remote log data synchronization.
</p>
### Remote log
### More on the remote log
The remote log lets receiving nodes know what data they are missing. Depending on the specific requirements and capabilities of the nodes and name system, the information can be referred to differently. We distinguish between three rough modes:
@ -74,8 +72,6 @@ The remote log lets receiving nodes know what data they are missing. Depending o
2. Normal sized page with CAS mapping
3. "Linked list" mode - minimally sized page with CAS mapping
**Data format:**
A remote log is a simply mapping from message identifiers to their corresponding address in a CAS.
| Message Identifier (H1) | CAS Hash (H2) |
@ -86,8 +82,7 @@ A remote log is a simply mapping from message identifiers to their corresponding
| | |
| *address to next page* |
Optionally, the content itself can be included, just like it normally would be
sent over the wire.
The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.
| Message Identifier (H1) | Content |
| ---------------- |---------------|
@ -97,32 +92,8 @@ sent over the wire.
| | |
| *address to next page* |
Here the upper section indicates a list of ordered pairs, and the lower section contains the address for the next page chunk. `H1` is the native hash function, and `H2` is the one used by the CAS. The numbers corresponds to the messages. To indicate which CAS is used, a remote log SHOULD use a multiaddr.
**Embedded data:**
A remote log MAY also choose to embed the wire payloads that corresponds to the native hash. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.
```
| H1_3 | | C_3 |
| H1_2 | | C_2 |
| H1_1 | | C_1 |
| -------------|
| next_page |
```
Here `C` stands for the content that would be stored at the CAS.
Both patterns can be used in parallel, e,g. by storing the last `k` messages directly and use CAS pointers for the rest. Together with the `next_page` page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.
### Next page semantics
The pointer to the 'next page' is another remote log entry, at a previous point
in time.
<!-- TODO: Determine requirement re overlapping, adjacent, and/or missing entries -->
### Interaction with MVDS
[vac.mvds.Message](https://specs.vac.dev/mvds.html#payloads) payloads are the only payloads that MUST be uploaded. Other messages types MAY be uploaded, depending on the implementation.