5.3 KiB

Raw Blame History

[WIP] vac.comm aka waku/next

This document is an early work in progress, and largely consists of notes that will be used to fill out a more thorough document such as [WIP] Dagger: A Distributed Storage Network and/or vacp2p.Communication - System Design & Technical Architecture.

vac.comm provides private and decentralized communication, and is part of the Vac communication/storage/compute umbrella.

Motivation and challenges

Challenges being addressed:

scalability of the network
incentived infrastructure and spam-resistance
build with resource restricted devices in mind, including nodes being mostly offline

System design, technical architecture and rationale

These are rough notes of things we currently know and don't know. They should be reformulated as a more coherent narrative.

Relation between storage and communication:

We'll use the same building blocks (Nim, libp2p and many its more basic protocols, etc)
We don't want to force a solution that fits for communication but not storage, and vice versa
That said, we'll continuously look and strive for overlap between the two when it comes to routing, incentivization, etc

On message routing:

Current Whisper-based routing fundamentally doesn't scale
For structured approaches, Kademlia for full nodes is the leading candidate
For unstructured approaches, gossipsub and episub are worth looking into more as complementary approaches

On Kademlia routing:

This provides point to point and point to neighborhood capabilities
Neighborhood routing is provided for with partial addressing a la PSS, and needs more experimental research to confirm feasibility
As a fallback, an unstructured gossip mechanism can be used for neighborhood messaging

On classical vs forwarding Kademlia:

Trade-off between classical and forwarding Kademlia: what's faster, opening new connections or re-using them?
Classical Kademlia better studied, and also more tractable when it comes to who is doing the work
We can likely provide both as options to keep flexibility, e.g. for heavy TCP forwarding might make more sense

On incentivization:

Structured and more direct approaches like classical Kademlia more tractable in terms of who does what
Incentivization in an unstructured routing setup a la gossip currently unproven
More ideas need to be explored and falsified aggressively in this area, e.g. see Block.Science effort, Swarm postage stamps, etc

On adaptive nodes:

Nodes with limited battery and short connection windows won't relay messages, and will instead likely rely on an emergent service network
In this emergent service network, lighter nodes can have a simple cryptoeconomic game with fuller nodes in a simple request/response manner
For semi-powered nodes, they can seamlessly join the DHT (e.g.) and provide some form of routing/storage

On security, spam protection and availability:

S/Kademlia can be used as anextension with some form of registration cost
Spam protection mechanisms needs further experimentation, e.g. stake based priority queue, postage stamps, zkSnarks rate limiting
Node uptime assumptions and replication factors are fairly well-studied in Kademlia and can be leveraged statistically

On privacy:

Transport privacy will be developed as a modular approach since not all use cases require it, and there are fundamental performance limitations
With pseudoanonymity, spam protection and E2EE a lot of privacy guarantees are achieved over current state of things
Offloading a lot of research to Nym et al, can also be complemented by making it easy to run over existing mixnet/Tor-like stacks, a la Briar

Timeline and milestones

February - March 2020

Ongoing:

Write an experimental Kademlia implementation for nim-libp2p, with the goal of

increasing tacit knowledge of Kademlia/libp2p/Nim
acting as a base layer for upcoming applied research work

Do research in conjunction with Dagger priorities as outlined below

Outstanding:

Get consensus on strategy, direction for most immediate users (Vac meetup, Dagger and Status Core)
Create a detailed timeline for implementing the project as a deliverable

Overlap / in conjunction with Dagger priorities:

Research existing projects such as: Swarm IPFS/Filecoin BitTorrent
Research relevant academic literature
Validating existing economic models as well as exploring our own

March - June 2020

As an initial goal, we will focus on building an MVP with a limited subset of features such as:

Send messages to a single point
Send messages to a neighborhood
Find nodes and content

This will require a few components in place:

Completed:

p2p networking stack we have an initial implementation of libp2p in Nim, this should be enough to start development, but it will be improved and augmented with new features on an ongoing basis