vacp2p
/
vac.dev-experimental-old
mirror of https://github.com/vacp2p/vac.dev-experimental-old.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Change post category

This commit is contained in:
Maria Rushkova 2021-08-10 12:07:34 +02:00 committed by Jakub Sokołowski
parent 3b60c4c1c7
commit 967c77e73a
No known key found for this signature in database
GPG Key ID: 4EF064D0E6D63020
19 changed files with 260 additions and 259 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
_posts/2019-07-19-p2p-data-sync-for-mobile.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "P2P Data Sync for Mobile" name: 'P2P Data Sync for Mobile'
title: "P2P Data Sync for Mobile" title: 'P2P Data Sync for Mobile'
date: 2019-07-19 12:00:00 +0800 date: 2019-07-19 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /p2p-data-sync-for-mobile permalink: /p2p-data-sync-for-mobile
categories: research category: research
summary: A research log. Reliable and decentralized, pick two. summary: A research log. Reliable and decentralized, pick two.
image: /assets/img/mvds_interactive.png image: /assets/img/mvds_interactive.png
--- ---
@ -22,14 +22,17 @@ There are many synchronization protocols out there and I won't go into detail of
Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices. Why do we want to do p2p sync for mobilephones in the first place? There are three components to that question. One is on the value of decentralization and peer-to-peer, the second is on why we'd want to reliably sync data at all, and finally why mobilephones and other resource restricted devices.
### Why p2p? ### Why p2p?
For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from. For decentralization and p2p, there are both technical and social/philosophical reasons. Technically, having a user-run network means it can scale with the number of users. Data locality is also improved if you query data that's close to you, similar to distributed CDNs. The throughput is also improved if there are more places to get data from.
Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat. Socially and philosophically, there are several ways to think about it. Open and decentralized networks also relate to the idea of open standards, i.e. compare the longevity of AOL with IRC or Bittorrent. One is run by a company and is shut down as soon as it stops being profitable, the others live on. Additionally increasingly control of data and infrastructure is becoming a liability. By having a network with no one in control, everyone is. It's ultimately a form of democratization, more similar to organic social structures pre Big Internet companies. This leads to properties such as censorship resistance and coercion resistance, where we limit the impact a 3rd party might have a voluntary interaction between individuals or a group of people. Examples of this are plentiful in the world of Facebook, Youtube, Twitter and WeChat.
### Why reliably sync data? ### Why reliably sync data?
At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on. At risk of stating the obvious, reliably syncing data is a requirement for many problem domains. You don't get this by default in a p2p world, as it is unreliable with nodes permissionslessly join and leave the network. In some cases you can get away with only ephemeral data, but usually you want some kind of guarantees. This is a must for reliable group chat experience, for example, where messages are expected to arrive in a timely fashion and in some reasonable order. The same is true for messages there represent financial transactions, and so on.
### Why mobilephones? ### Why mobilephones?
Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence. Most devices people use daily are mobile phones. It's important to provide the same or at least similar guarantees to more traditional p2p nodes that might run on a desktop computer or computer. The alternative is to rely on gateways, which shares many of the drawbacks of centralized control and prone to censorship, control and surveillence.
More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc. More generally, resource restricted devices can differ in their capabilities. One example is smartphones, but others are: desktop, routers, Raspberry PIs, POS systems, and so on. The number and diversity of devices are exploding, and it's useful to be able to leverage this for various types of infrastructure. The alternative is to centralize on big cloud providers, which also lends itself to lack of democratization and censorship, etc.
@ -90,18 +93,19 @@ Each failed attempt implies another retransmission. That means we have `(1/0.1)^
The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost. The problem above hints at the requirements 3 and 4 above. While we did get reliable syncing (requirement 1), it came at a big cost.
There are a few ways of getting around this issue. One is having a *store and forward* model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status. There are a few ways of getting around this issue. One is having a _store and forward_ model, where some intermediary node picks up (encrypted) messages and forwards them to the recipient. This is what we have in production right now at Status.
Another, arguably more pure and robust, way is having a *remote log*, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS. Another, arguably more pure and robust, way is having a _remote log_, where the actual data is spread over some decentralized storage layer, and you have a mutable reference to find the latest messages, similar to DNS.
What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are *required* to get reliable data transmission. What they both have in common is that they act as a sort of highly-available cache to smooth over the non-overlapping connection windows between two endpoints. Neither of them are _required_ to get reliable data transmission.
### Basic calculations for bandwidth multiplier ### Basic calculations for bandwidth multiplier
While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions: While we do want better simulations, and this is a work in progress, we can also look at the above scenarios using some basic calculations. This allows us to build a better intuition and reason about the problem without having to write code. Let's start with some assumptions:
- two nodes exchanging a single message in batch mode - two nodes exchanging a single message in batch mode
- 10% uniformly random uptime for each node - 10% uniformly random uptime for each node
- in HA cache case, 100% uptime of a piece of infrastructure C - in HA cache case, 100% uptime of a piece of infrastructure C
- retransmission every epoch (with constant or exponential backoff) - retransmission every epoch (with constant or exponential backoff)
- only looking at average (p50) case - only looking at average (p50) case
@ -116,7 +120,7 @@ A <- ack B (10% chance of arrival)
With a constant backoff, A will send messages at epoch `1, 2, 3, ...`. With exponential backoff and a multiplier of 2, this would be `1, 2, 4, 8, ...`. Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier. With a constant backoff, A will send messages at epoch `1, 2, 3, ...`. With exponential backoff and a multiplier of 2, this would be `1, 2, 4, 8, ...`. Let's assume constant backoff for now, as this is what will influence the success rate and thus the bandwidth multiplier.
There's a difference between *time to receive* and *time to stop sending*. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore: There's a difference between _time to receive_ and _time to stop sending_. Assuming each send attempt is independent, it takes on average 10 epochs for A's message to arrive with B. Furthermore:
1. A will send messages until it receives an ACK. 1. A will send messages until it receives an ACK.
2. B will send ACK if it receives a message. 2. B will send ACK if it receives a message.

41
_posts/2019-08-02-vac-overview.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "Vac - A Rough Overview" name: 'Vac - A Rough Overview'
title: "Vac - A Rough Overview" title: 'Vac - A Rough Overview'
date: 2019-08-02 12:00:00 +0800 date: 2019-08-02 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /vac-overview permalink: /vac-overview
categories: research category: research
summary: Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems. summary: Vac is a modular peer-to-peer messaging stack, with a focus on secure messaging. Overview of terms, stack and open problems.
--- ---
@ -14,31 +14,31 @@ Vac is a **modular peer-to-peer messaging stack, with a focus on secure messagin
## Basic terms ## Basic terms
*messaging stack*. While the initial focus is on [data sync](https://vac.dev/p2p-data-sync-for-mobile), we are concerned with all layers in the stack. That means all the way from underlying transports, p2p overlays and routing, to initial trust establishment and semantics for things like group chat. The ultimate goal is to give application developers the tools they need to provide secure messaging for their users, so they can focus on their domain expertise. _messaging stack_. While the initial focus is on [data sync](https://vac.dev/p2p-data-sync-for-mobile), we are concerned with all layers in the stack. That means all the way from underlying transports, p2p overlays and routing, to initial trust establishment and semantics for things like group chat. The ultimate goal is to give application developers the tools they need to provide secure messaging for their users, so they can focus on their domain expertise.
*modular*. Unlike many other secure messaging applications, our goal is not to have a tightly coupled set of protocols, nor is it to reinvent the wheel. Instead, we aim to provide options at each layer in the stack, and build on the shoulders of giants, putting a premimum on interoperability. It's similar in philosophy to projects such as [libp2p](https://libp2p.io/) or [Substrate](https://www.parity.io/substrate/) in that regard. Each choice comes with different trade-offs, and these look different for different applications. _modular_. Unlike many other secure messaging applications, our goal is not to have a tightly coupled set of protocols, nor is it to reinvent the wheel. Instead, we aim to provide options at each layer in the stack, and build on the shoulders of giants, putting a premimum on interoperability. It's similar in philosophy to projects such as [libp2p](https://libp2p.io/) or [Substrate](https://www.parity.io/substrate/) in that regard. Each choice comes with different trade-offs, and these look different for different applications.
*peer-to-peer*. The protocols we work on are pure p2p, and aim to minimize centralization. This too is in opposition to many initiatives in the secure messaging space. _peer-to-peer_. The protocols we work on are pure p2p, and aim to minimize centralization. This too is in opposition to many initiatives in the secure messaging space.
*messaging*. By messaging we mean messaging in a generalized sense. This includes both human to human communication, as well machine to machine communication. By messaging we also mean something more fundamental than text messages, we also include things like transactions (state channels, etc) under this moniker. _messaging_. By messaging we mean messaging in a generalized sense. This includes both human to human communication, as well machine to machine communication. By messaging we also mean something more fundamental than text messages, we also include things like transactions (state channels, etc) under this moniker.
*secure messaging*. Outside of traditional notions of secure messaging, such as ensuring end to end encryption, forward secrecy, avoiding MITM-attacks, etc, we are also concerned with two other forms of secure messaging. We call these *private messaging* and *censorship-resistance*. Private messaging means viewing privacy as a security property, with all that entails. Censorship resistance ties into being p2p, but also in terms of allowing for transports and overlays that can't easily be censored by port blocking, traffic analysis, and similar. _secure messaging_. Outside of traditional notions of secure messaging, such as ensuring end to end encryption, forward secrecy, avoiding MITM-attacks, etc, we are also concerned with two other forms of secure messaging. We call these _private messaging_ and _censorship-resistance_. Private messaging means viewing privacy as a security property, with all that entails. Censorship resistance ties into being p2p, but also in terms of allowing for transports and overlays that can't easily be censored by port blocking, traffic analysis, and similar.
*Vāc*. Is a Vedic goddess of speech. It also hints at being a vaccine. _Vāc_. Is a Vedic goddess of speech. It also hints at being a vaccine.
## Protocol stack ## Protocol stack
What does this stack look like? We take inspiration from [core](https://tools.ietf.org/html/rfc793) [internet architecture](https://www.ietf.org/rfc/rfc1122.txt), existing [survey work](http://cacr.uwaterloo.ca/techreports/2015/cacr2015-02.pdf) and other [efforts](https://code.briarproject.org/briar/briar/wikis/A-Quick-Overview-of-the-Protocol-Stack) that have been done to decompose the problem into orthogonal pieces. Each layer provides their own set of properties and only interact with the layers it is adjacent to. Note that this is a rough sketch. What does this stack look like? We take inspiration from [core](https://tools.ietf.org/html/rfc793) [internet architecture](https://www.ietf.org/rfc/rfc1122.txt), existing [survey work](http://cacr.uwaterloo.ca/techreports/2015/cacr2015-02.pdf) and other [efforts](https://code.briarproject.org/briar/briar/wikis/A-Quick-Overview-of-the-Protocol-Stack) that have been done to decompose the problem into orthogonal pieces. Each layer provides their own set of properties and only interact with the layers it is adjacent to. Note that this is a rough sketch.
| Layer / Protocol | Purpose | Examples | | Layer / Protocol | Purpose | Examples |
|-------------------|-----------------------------------|----------------------| | ------------------- | --------------------------------- | -------------------- |
| Application layer | End user semantics | 1:1 chat, group chat | | Application layer | End user semantics | 1:1 chat, group chat |
| Data Sync | Data consistency | MVDS, BSP | | Data Sync | Data consistency | MVDS, BSP |
| Secure Transport | Confidentiality, PFS, etc | Double Ratchet, MLS | | Secure Transport | Confidentiality, PFS, etc | Double Ratchet, MLS |
| Transport Privacy | Transport and metadata protection | Whisper, Tor, Mixnet | | Transport Privacy | Transport and metadata protection | Whisper, Tor, Mixnet |
| P2P Overlay | Overlay routing, NAT traversal | devp2p, libp2p | | P2P Overlay | Overlay routing, NAT traversal | devp2p, libp2p |
| | | | | |
| Trust Establishment | Establishing end-to-end trust | TOFU, web of trust | | Trust Establishment | Establishing end-to-end trust | TOFU, web of trust |
As an example, end user semantics such as group chat or moderation capabilities can largely work regardless of specific choices further down the stack. Similarly, using a mesh network or Tor doesn't impact the use of Double Ratchet at the Secure Transport layer. As an example, end user semantics such as group chat or moderation capabilities can largely work regardless of specific choices further down the stack. Similarly, using a mesh network or Tor doesn't impact the use of Double Ratchet at the Secure Transport layer.
@ -51,11 +51,13 @@ In terms of specific properties and trade-offs at each layer, we'll go deeper do
With all the pieces involved, this is quite an undertaking. Luckily, a lot of pieces are already in place and can be either incorporated as-is or iterated on. In terms of medium and long term, here's a rough sketch of priorities and open problems. With all the pieces involved, this is quite an undertaking. Luckily, a lot of pieces are already in place and can be either incorporated as-is or iterated on. In terms of medium and long term, here's a rough sketch of priorities and open problems.
1. **Better data sync.** While the current [MVDS](https://specs.vac.dev/specs/mvds.html) works, it is lacking in a few areas: 1. **Better data sync.** While the current [MVDS](https://specs.vac.dev/specs/mvds.html) works, it is lacking in a few areas:
- Lack of remote log for mostly-offline offline devices - Lack of remote log for mostly-offline offline devices
- Better scalability for multi-user chat contexts - Better scalability for multi-user chat contexts
- Better usability in terms of application layer usage and supporting more transports - Better usability in terms of application layer usage and supporting more transports
2. **Better transport layer support.** Currently MVDS runs primarily over Whisper, which has a few issues: 2. **Better transport layer support.** Currently MVDS runs primarily over Whisper, which has a few issues:
- scalability, being able to run with many nodes - scalability, being able to run with many nodes
- spam-resistance, proof of work is a poor mechanism for heterogeneous devices - spam-resistance, proof of work is a poor mechanism for heterogeneous devices
- no incentivized infrastructure, leading to centralized choke points - no incentivized infrastructure, leading to centralized choke points
@ -67,7 +69,6 @@ In addition to these most immediate concerns, there are other open problems. Som
4. **Incentivized and spam-resistant messaging.** Reasons to run infrastructure and not relying on altruistic nodes. For spam resistance, in p2p multicast spam is a big attack vector due to amplification. There are a few interesting directions here, such as EigenTrust, proof of burn with micropayments, and leveraging zero-knowledge proofs. 4. **Incentivized and spam-resistant messaging.** Reasons to run infrastructure and not relying on altruistic nodes. For spam resistance, in p2p multicast spam is a big attack vector due to amplification. There are a few interesting directions here, such as EigenTrust, proof of burn with micropayments, and leveraging zero-knowledge proofs.
5. **Strong privacy guarantees at transport privacy layer**. More rigorous privacy guarantees and explicit trade-offs for metadata protection. Includes Mixnet. 5. **Strong privacy guarantees at transport privacy layer**. More rigorous privacy guarantees and explicit trade-offs for metadata protection. Includes Mixnet.
6. **Censorship-resistant and robust P2P overlay**. NAT traversal; running in the browser; mesh networks; pluggable transports for traffic obfuscation. 6. **Censorship-resistant and robust P2P overlay**. NAT traversal; running in the browser; mesh networks; pluggable transports for traffic obfuscation.
7. **Scalable and decentralized secure conversational security.** Strong security guarantees such as forward secrecy, post compromise security, for large group chats. Includes projects such MLS and extending Double Ratchet. 7. **Scalable and decentralized secure conversational security.** Strong security guarantees such as forward secrecy, post compromise security, for large group chats. Includes projects such MLS and extending Double Ratchet.

60
_posts/2019-10-04-remote-log.md
View File

@ -1,21 +1,21 @@
--- ---
layout: post layout: post
name: "P2P Data Sync with a Remote Log" name: 'P2P Data Sync with a Remote Log'
title: "P2P Data Sync with a Remote Log" title: 'P2P Data Sync with a Remote Log'
date: 2019-10-04 12:00:00 +0800 date: 2019-10-04 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /remote-log permalink: /remote-log
categories: research category: research
summary: A research log. Asynchronous P2P messaging? Remote logs to the rescue! summary: A research log. Asynchronous P2P messaging? Remote logs to the rescue!
image: /assets/img/remote-log.png image: /assets/img/remote-log.png
--- ---
A big problem when doing end-to-end data sync between mobile nodes is that most devices are offline most of the time. With a naive approach, you quickly run into issues of 'ping-pong' behavior, where messages have to be constantly retransmitted. We saw some basic calculations of what this bandwidth multiplier looks like in a [previous post](https://vac.dev/p2p-data-sync-for-mobile). A big problem when doing end-to-end data sync between mobile nodes is that most devices are offline most of the time. With a naive approach, you quickly run into issues of 'ping-pong' behavior, where messages have to be constantly retransmitted. We saw some basic calculations of what this bandwidth multiplier looks like in a [previous post](https://vac.dev/p2p-data-sync-for-mobile).
While you could do some background processing, this is really battery-draining, and on iOS these capabilities are limited. A better approach instead is to loosen the constraint that two nodes need to be online at the same time. How do we do this? There are two main approaches, one is the *store and forward model*, and the other is a *remote log*. While you could do some background processing, this is really battery-draining, and on iOS these capabilities are limited. A better approach instead is to loosen the constraint that two nodes need to be online at the same time. How do we do this? There are two main approaches, one is the _store and forward model_, and the other is a _remote log_.
In the *store and forward* model, we use an intermediate node that forward messages on behalf of the recipient. In the *remote log* model, you instead replicate the data onto some decentralized storage, and have a mutable reference to the latest state, similar to DNS. While both work, the latter is somewhat more elegant and "pure", as it has less strict requirements of an individual node's uptime. Both act as a highly-available cache to smoothen over non-overlapping connection windows between endpoints. In the _store and forward_ model, we use an intermediate node that forward messages on behalf of the recipient. In the _remote log_ model, you instead replicate the data onto some decentralized storage, and have a mutable reference to the latest state, similar to DNS. While both work, the latter is somewhat more elegant and "pure", as it has less strict requirements of an individual node's uptime. Both act as a highly-available cache to smoothen over non-overlapping connection windows between endpoints.
In this post we are going to describe how such a remote log schema could work. Specifically, how it enhances p2p data sync and takes care of the [following requirements](https://vac.dev/p2p-data-sync-for-mobile): In this post we are going to describe how such a remote log schema could work. Specifically, how it enhances p2p data sync and takes care of the [following requirements](https://vac.dev/p2p-data-sync-for-mobile):
@ -31,15 +31,15 @@ In this post we are going to describe how such a remote log schema could work. S
A remote log is a replication of a local log. This means a node can read data from a node that is offline. A remote log is a replication of a local log. This means a node can read data from a node that is offline.
The spec is in an early draft stage and can be found [here](https://github.com/vacp2p/specs/pull/16). A very basic [spike](https://en.wikipedia.org/wiki/Spike_(software_development)) / proof-of-concept can be found [here](https://github.com/vacp2p/research/tree/master/remote_log). The spec is in an early draft stage and can be found [here](https://github.com/vacp2p/specs/pull/16). A very basic [spike](<https://en.wikipedia.org/wiki/Spike_(software_development)>) / proof-of-concept can be found [here](https://github.com/vacp2p/research/tree/master/remote_log).
### Definitions ### Definitions
| Term | Definition | | Term | Definition |
| ----------- | -------------------------------------------------------------------------------------- | | ---------- | ------------------------------------------------------------------------- |
| CAS | Content-addressed storage. Stores data that can be addressed by its hash. | | CAS | Content-addressed storage. Stores data that can be addressed by its hash. |
| NS | Name system. Associates mutable data to a name. | | NS | Name system. Associates mutable data to a name. |
| Remote log | Replication of a local log at a different location. | | Remote log | Replication of a local log at a different location. |
### Roles ### Roles
@ -47,10 +47,10 @@ There are four fundamental roles:
1. Alice 1. Alice
2. Bob 2. Bob
2. Name system (NS) 3. Name system (NS)
3. Content-addressed storage (CAS) 4. Content-addressed storage (CAS)
The *remote log* is the data format of what is stored in the name system. The _remote log_ is the data format of what is stored in the name system.
"Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS. "Bob" can represent anything from 0 to N participants. Unlike Alice, Bob only needs read-only access to NS and CAS.
@ -75,22 +75,22 @@ The remote log lets receiving nodes know what data they are missing. Depending o
A remote log is simply a mapping from message identifiers to their corresponding address in a CAS: A remote log is simply a mapping from message identifiers to their corresponding address in a CAS:
| Message Identifier (H1) | CAS Hash (H2) | | Message Identifier (H1) | CAS Hash (H2) |
| ---------------- |---------------| | ----------------------- | ------------- |
| H1_3 | H2_3 | | H1_3 | H2_3 |
| H1_2 | H2_2 | | H1_2 | H2_2 |
| H1_1 | H2_1 | | H1_1 | H2_1 |
| | | | | |
| *address to next page* | | _address to next page_ |
The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage. The numbers here corresponds to messages. Optionally, the content itself can be included, just like it normally would be sent over the wire. This bypasses the need for a dedicated CAS and additional round-trips, with a trade-off in bandwidth usage.
| Message Identifier (H1) | Content | | Message Identifier (H1) | Content |
| ---------------- |---------------| | ----------------------- | ------- |
| H1_3 | C3 | | H1_3 | C3 |
| H1_2 | C2 | | H1_2 | C2 |
| H1_1 | C1 | | H1_1 | C1 |
| | | | | |
| *address to next page* | | _address to next page_ |
Both patterns can be used in parallel, e,g. by storing the last `k` messages directly and use CAS pointers for the rest. Together with the `next_page` page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage. Both patterns can be used in parallel, e,g. by storing the last `k` messages directly and use CAS pointers for the rest. Together with the `next_page` page semantics, this gives users flexibility in terms of bandwidth and latency/indirection, all the way from a simple linked list to a fully replicated log. The latter is useful for things like backups on durable storage.
@ -103,13 +103,13 @@ Both patterns can be used in parallel, e,g. by storing the last `k` messages dir
The spec is still in an early draft stage, so it is expected to change. Same with the proof of concept. More work is needed on getting a fully featured proof of concept with specific CAS and NAS instances. E.g. Swarm and Swarm Feeds, or IPFS and IPNS, or something else. The spec is still in an early draft stage, so it is expected to change. Same with the proof of concept. More work is needed on getting a fully featured proof of concept with specific CAS and NAS instances. E.g. Swarm and Swarm Feeds, or IPFS and IPNS, or something else.
For data sync in general: For data sync in general:
- Make consistency guarantees more explicit for app developers with support for sequence numbers and DAGs, as well as the ability to send non-synced messages. E.g. ephemeral typing notifications, linear/sequential history and casual consistency/DAG history
- Make consistency guarantees more explicit for app developers with support for sequence numbers and DAGs, as well as the ability to send non-synced messages. E.g. ephemeral typing notifications, linear/sequential history and casual consistency/DAG history
- Better semantics and scalability for multi-user sync contexts, e.g. CRDTs and joining multiple logs together - Better semantics and scalability for multi-user sync contexts, e.g. CRDTs and joining multiple logs together
- Better usability in terms of application layer usage (data sync clients) and supporting more transports - Better usability in terms of application layer usage (data sync clients) and supporting more transports
--- ---
PS1. Thanks everyone who submitted great [logo proposals](https://explorer.bounties.network/bounty/3389) for Vac! PS1. Thanks everyone who submitted great [logo proposals](https://explorer.bounties.network/bounty/3389) for Vac!
PPS2. Next week on October 10th decanus and I will be presenting Vac at [Devcon](https://devcon.org/agenda), come say hi :) PPS2. Next week on October 10th decanus and I will be presenting Vac at [Devcon](https://devcon.org/agenda), come say hi :)

33
_posts/2019-11-08-feasibility-semaphore-rate-limiting-zksnarks.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "Feasibility Study: Semaphore rate limiting through zkSNARKs" name: 'Feasibility Study: Semaphore rate limiting through zkSNARKs'
title: "Feasibility Study: Semaphore rate limiting through zkSNARKs" title: 'Feasibility Study: Semaphore rate limiting through zkSNARKs'
date: 2019-11-08 12:00:00 +0800 date: 2019-11-08 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /feasibility-semaphore-rate-limiting-zksnarks permalink: /feasibility-semaphore-rate-limiting-zksnarks
categories: research category: research
summary: A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks. summary: A research log. Zero knowledge signaling as a rate limiting mechanism to prevent spam in p2p networks.
image: /assets/img/peacock-signaling.jpg image: /assets/img/peacock-signaling.jpg
discuss: https://forum.vac.dev/t/discussion-feasibility-study-semaphore-rate-limiting-through-zksnarks/21 discuss: https://forum.vac.dev/t/discussion-feasibility-study-semaphore-rate-limiting-through-zksnarks/21
@ -33,13 +33,13 @@ In addition to above, there are a lot of related problems that share similaritie
## Basic terminology ## Basic terminology
A *zero-knowledge proof* allows a *prover* to show a *verifier* that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id. A _zero-knowledge proof_ allows a _prover_ to show a _verifier_ that they know something, without revealing what that something is. This means you can do trust-minimized computation that is also privacy preserving. As a basic example, instead of showing your ID when going to a bar you simply give them a proof that you are over 18, without showing the doorman your id.
*zkSNARKs* is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more. _zkSNARKs_ is a form of zero-knowledge proofs. There are many types of zero-knowledge proofs, and the field is evolving rapidly. They come with various trade-offs in terms of things such as: trusted setup, cryptographic assumptions, proof/verification key size, proof/verification time, proof size, etc. See section below for more.
*Semaphore* is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See [current Github home](https://github.com/kobigurk/semaphore) for more. _Semaphore_ is a framework/library/construct on top of zkSNARks. It allows for zero-knowledge signaling, specifically on top of Ethereum. This means an approved user can broadcast some arbitrary string without revealing their identity, given some specific constraints. An approved user is someone who has been added to a certain merkle tree. See [current Github home](https://github.com/kobigurk/semaphore) for more.
*Circom* is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See [Github](https://github.com/iden3/circom) for more. _Circom_ is a DSL for writing arithmetic circuits that can be used in zkSNARKs, similar to how you might write a NAND gate. See [Github](https://github.com/iden3/circom) for more.
## Basic flow ## Basic flow
@ -47,28 +47,23 @@ We start with a private voting example, and then extend it to the slashable rate
1. A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are. 1. A user registers an identity (arbitrary keypair), along with a small fee, to a smart contract. This adds them to a merkle tree and allows them to prove that they are member of that group, without revealing who they are.
2. When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some *public outputs*, and can be verified by anyone (including a smart contract). 2. When a user wants to send a message, they compute a zero-knowledge proof. This ensures certain invariants, have some _public outputs_, and can be verified by anyone (including a smart contract).
3. Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as "2020-01-01 vote on Foo Bar for president". 3. Any node can verify the proof, including smart contracts on chain (as of Byzantinum HF). Additionally, a node can have rules for the public output. In the case of voting, one such rule is that a specific output hash has to be equal to some predefined value, such as "2020-01-01 vote on Foo Bar for president".
4. Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once. 4. Because of how the proof is constructed, and the rules around output values, this ensures that: a user is part of the approved set of voters and that a user can only vote once.
5. As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified. 5. As a consequence of above, we have a system where registered users can only vote once, no one can see who voted for what, and this can all be proven and verified.
### Rate limiting example ### Rate limiting example
In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat. In the case of rate limiting, we do want nodes to send multiple messages. This changes step 3-5 above somewhat.
*NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same*. _NOTE: It is a bit more involved than this, and if we precompute proofs the flow might look a bit different. But the general idea is the same_.
1. Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s. 1. Instead of having a rule that you can only vote once, we have a rule that you can only send a message per epoch. Epoch here can be every second, as defined by UTC date time +-20s.
2. Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key). 2. Additionally, if a users sends more than one message per epoch, one of the public outputs is a random share of a private key. Using Shamir's Secret Sharing (similar to a multisig) and 2/3 key share as an example threshold: in the normal case only 1/3 private keys is revealed, which is insufficient to have access. In the case where two messages are sent in an epoch, probabilistically 2/3 shares is sufficient to have access to the key (unless you get the same random share of the key).
3. This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them. 3. This means any untrusted user who detects a spamming user, can use it to access their private key corresponding to funds in the contract, and thus slash them.
4. As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it. 4. As a consequence of above, we have a system where registered users can only messages X times per epoch, and no one can see who is sending what messages. Additionally, if a user is violating the above rate limit, they can be punished and any user can profit from it.
### Briefly on scope of 'approved users' ### Briefly on scope of 'approved users'
In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration. In the case of an application like Status, this construct can either be a global StatusNetwork group, or one per chat, or network, etc. It can be applied both at the network and user level. There are no specific limitations on where or who deploys this, and it is thus more of a UX consideration.
@ -119,7 +114,7 @@ See [details](https://github.com/vacp2p/research/issues/11).
#### Licensing issues #### Licensing issues
Currently Circom [uses a GPL license](https://github.com/iden3/circom/blob/master/COPYING ), which can get tricky when it comes to the App Store etc. Currently Circom [uses a GPL license](https://github.com/iden3/circom/blob/master/COPYING), which can get tricky when it comes to the App Store etc.
See [details](https://github.com/vacp2p/research/issues/12). See [details](https://github.com/vacp2p/research/issues/12).
@ -149,4 +144,4 @@ For example, this might also include leveraging largely ready made solutions suc
Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues. Thanks to Barry Whitehat for patient explanation and pointers. Thanks to WJ for helping with runtime issues.
*Peacock header image from [Tonos](https://en.wikipedia.org/wiki/File:Flickr_-_lo.tangelini_-_Tonos_(1).jpg).* _Peacock header image from [Tonos](<https://en.wikipedia.org/wiki/File:Flickr_-_lo.tangelini_-_Tonos_(1).jpg>)._

12
_posts/2019-12-03-fixing-whisper-with-waku.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "Fixing Whisper with Waku" name: 'Fixing Whisper with Waku'
title: "Fixing Whisper with Waku" title: 'Fixing Whisper with Waku'
date: 2019-12-03 12:00:00 +0800 date: 2019-12-03 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /fixing-whisper-with-waku permalink: /fixing-whisper-with-waku
categories: research category: research
summary: A research log. Why Whisper doesn't scale and how to fix it. summary: A research log. Why Whisper doesn't scale and how to fix it.
image: /assets/img/whisper_scalability.png image: /assets/img/whisper_scalability.png
discuss: https://forum.vac.dev/t/discussion-fixing-whisper-with-waku/27 discuss: https://forum.vac.dev/t/discussion-fixing-whisper-with-waku/27
@ -41,7 +41,7 @@ In this post, we'll focus on the first problem, which is scalability through ban
## Whisper theoretical scalability model ## Whisper theoretical scalability model
*(Feel free to skip this section if you want to get right to the results).* _(Feel free to skip this section if you want to get right to the results)._
There's widespread implicit knowledge that Whisper "doesn't scale", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see [Status Whisper usage There's widespread implicit knowledge that Whisper "doesn't scale", but it is less understood exactly why. This theoretical model attempts to encode some characteristics of it. Specifically for use case such as one by Status (see [Status Whisper usage
spec](https://github.com/status-im/specs/blob/master/status-whisper-usage-spec.md)). spec](https://github.com/status-im/specs/blob/master/status-whisper-usage-spec.md)).
@ -289,6 +289,6 @@ iterative work with results on the order of weeks.
In short, we have a [Waku version 0 spec up](https://specs.vac.dev/specs/waku.html) as well as a [PoC](https://github.com/status-im/nim-eth/pull/120) for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for [Waku mode](https://github.com/status-im/nim-eth/pull/114). See [rough roadmap](https://github.com/vacp2p/pm/issues/5), [project board](https://github.com/orgs/vacp2p/projects/2) and progress thread on the [Vac forum](https://forum.vac.dev/t/waku-project-and-progress/24). In short, we have a [Waku version 0 spec up](https://specs.vac.dev/specs/waku.html) as well as a [PoC](https://github.com/status-im/nim-eth/pull/120) for backwards compatibility. In the coming weeks, we are going to solidify the specs, get a more fully featured PoC for [Waku mode](https://github.com/status-im/nim-eth/pull/114). See [rough roadmap](https://github.com/vacp2p/pm/issues/5), [project board](https://github.com/orgs/vacp2p/projects/2) and progress thread on the [Vac forum](https://forum.vac.dev/t/waku-project-and-progress/24).
The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously [ad hoc implemented features](https://specs.vac.dev/specs/waku.html#additional-capabilities), such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the `geth` (Go), `status/whisper` (Go) and `nim-eth` (Nim) versions, specifically around light node usage and the handshake. The spec has been rewrittten for clarity, with ABNF grammar and less ambiguous language. The spec also incorporates several previously [ad hoc implemented features](https://specs.vac.dev/specs/waku.html#additional-capabilities), such as light nodes and mailserver/client support. This has already caught a few incompatibilities between the `geth` (Go), `status/whisper` (Go) and `nim-eth` (Nim) versions, specifically around light node usage and the handshake.
If you are interested in this effort, please check out [our forum](https://forum.vac.dev/) for questions, comments and proposals. We already have some discussion for better [spam protection](https://forum.vac.dev/t/stake-priority-based-queuing/26) (see [previous post](https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks) for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement. If you are interested in this effort, please check out [our forum](https://forum.vac.dev/) for questions, comments and proposals. We already have some discussion for better [spam protection](https://forum.vac.dev/t/stake-priority-based-queuing/26) (see [previous post](https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks) for a more complex but privacy-preserving proposal), something that is likely going to be addressed in future versions of Waku, along with many other fixes and enhancement.

52
_posts/2020-02-14-waku-update.md
View File

@ -1,18 +1,18 @@
--- ---
layout: post layout: post
name: "Waku Update" name: 'Waku Update'
title: "Waku Update" title: 'Waku Update'
date: 2020-02-14 12:00:00 +0800 date: 2020-02-14 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /waku-update permalink: /waku-update
categories: research category: research
summary: A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next? summary: A research log. What's the current state of Waku? How many users does it support? What are the bottlenecks? What's next?
image: /assets/img/waku_infrastructure_sky.jpg image: /assets/img/waku_infrastructure_sky.jpg
discuss: https://forum.vac.dev/t/waku-update-where-are-we-at/34 discuss: https://forum.vac.dev/t/waku-update-where-are-we-at/34
--- ---
Waku is our fork of Whisper where we address the shortcomings of Whisper in an iterative manner. We've seen a in [previous post](https://vac.dev/fixing-whisper-with-waku) that Whisper doesn't scale, and why. In this post we'll talk about what the current state of Waku is, how many users it can support, and future plans. Waku is our fork of Whisper where we address the shortcomings of Whisper in an iterative manner. We've seen a in [previous post](https://vac.dev/fixing-whisper-with-waku) that Whisper doesn't scale, and why. In this post we'll talk about what the current state of Waku is, how many users it can support, and future plans.
## Current state ## Current state
@ -51,12 +51,13 @@ We have a [simulation](https://github.com/status-im/nimbus/tree/master/waku#test
This is our current understanding of how many users a network running Waku can support. Specifically in the context of the Status chat app, since that's the most immediate consumer of Waku. It should generalize fairly well to most deployments. This is our current understanding of how many users a network running Waku can support. Specifically in the context of the Status chat app, since that's the most immediate consumer of Waku. It should generalize fairly well to most deployments.
**tl;dr (for Status app):** **tl;dr (for Status app):**
- beta: 100 DAU - beta: 100 DAU
- v1: 1k DAU - v1: 1k DAU
- v1.1 (waku only): 10k DAU (up to x10 with deployment hotfixes) - v1.1 (waku only): 10k DAU (up to x10 with deployment hotfixes)
- v1.2 (waku+dns): 100k DAU (can optionally be folded into v1.1) - v1.2 (waku+dns): 100k DAU (can optionally be folded into v1.1)
*Assuming 10 concurrent users = 100 DAU. Estimate uncertainty increases for each order of magnitude until real-world data is observed.* _Assuming 10 concurrent users = 100 DAU. Estimate uncertainty increases for each order of magnitude until real-world data is observed._
As far as we know right now, these are the bottlenecks we have: As far as we know right now, these are the bottlenecks we have:
@ -83,30 +84,31 @@ For light node, bloom filter is set to almost 10% false positive (bloom filter:
**Star network:** **Star network:**
| Description | Peers | Valid | Invalid | | Description | Peers | Valid | Invalid |
|-----------------|-------|-------|---------| | --------------- | ----- | ----- | ------- |
| Master node | 7 | 10001 | 0 | | Master node | 7 | 10001 | 0 |
| Full node 1 | 3 | 10001 | 0 | | Full node 1 | 3 | 10001 | 0 |
| Full node 2 | 1 | 10001 | 0 | | Full node 2 | 1 | 10001 | 0 |
| Full node 3 | 1 | 10001 | 0 | | Full node 3 | 1 | 10001 | 0 |
| Full node 4 | 1 | 10001 | 0 | | Full node 4 | 1 | 10001 | 0 |
| Full node 5 | 1 | 10001 | 0 | | Full node 5 | 1 | 10001 | 0 |
| Light node | 2 | 815 | 0 | | Light node | 2 | 815 | 0 |
| Waku light node | 2 | 1 | 0 | | Waku light node | 2 | 1 | 0 |
**Full mesh:** **Full mesh:**
| Description | Peers | Valid | Invalid | | Description | Peers | Valid | Invalid |
|-----------------|-------|-------|---------| | --------------- | ----- | ----- | ------- |
| Full node 0 | 7 | 10001 | 20676 | | Full node 0 | 7 | 10001 | 20676 |
| Full node 1 | 7 | 10001 | 9554 | | Full node 1 | 7 | 10001 | 9554 |
| Full node 2 | 5 | 10001 | 23304 | | Full node 2 | 5 | 10001 | 23304 |
| Full node 3 | 5 | 10001 | 11983 | | Full node 3 | 5 | 10001 | 11983 |
| Full node 4 | 5 | 10001 | 24425 | | Full node 4 | 5 | 10001 | 24425 |
| Full node 5 | 5 | 10001 | 23472 | | Full node 5 | 5 | 10001 | 23472 |
| Light node | 2 | 803 | 803 | | Light node | 2 | 803 | 803 |
| Waku light node | 2 | 1 | 1 | | Waku light node | 2 | 1 | 1 |
Things to note: Things to note:
- Whisper light node with ~10% false positive gets ~10% of total traffic - Whisper light node with ~10% false positive gets ~10% of total traffic
- Waku light node gets ~1000x less envelopes than Whisper light node - Waku light node gets ~1000x less envelopes than Whisper light node
- Full mesh results in a lot more duplicate messages, expect for Waku light node - Full mesh results in a lot more duplicate messages, expect for Waku light node
@ -141,4 +143,4 @@ For the third bottleneck, a likely candidate for fixing this is Kademlia routing
## Acknowledgements ## Acknowledgements
*Image from "caged sky" by mh.xbhd.org is licensed under CC BY 2.0 (https://ccsearch.creativecommons.org/photos/a9168311-78de-4cb7-a6ad-f92be8361d0e)* _Image from "caged sky" by mh.xbhd.org is licensed under CC BY 2.0 (https://ccsearch.creativecommons.org/photos/a9168311-78de-4cb7-a6ad-f92be8361d0e)_

23
_posts/2020-02-7-dns-based-discovery.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "DNS Based Discovery" name: 'DNS Based Discovery'
title: "DNS Based Discovery" title: 'DNS Based Discovery'
date: 2020-02-7 12:00:00 +0100 date: 2020-02-7 12:00:00 +0100
author: dean author: dean
published: true published: true
permalink: /dns-based-discovery permalink: /dns-based-discovery
categories: research category: research
summary: A look at EIP-1459 and the benefits of DNS based discovery. summary: A look at EIP-1459 and the benefits of DNS based discovery.
--- ---
@ -16,11 +16,11 @@ Another issue is how we do the initial bootstrapping. In other words, how does a
In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable. In summary, we have a static list that provides us with nodes we can connect to which then allows us to discover the rest of the network using something like Kademlia. But what we need is something that can easily be mutated, guarantees a certain amount of security, and is efficient for resource restricted devices. Ideally our solution would also be robust and scalable.
How do we do this? How do we do this?
[EIP 1459: Node Discovery via DNS](https://eips.ethereum.org/EIPS/eip-1459), which is one of the strategies we are using for discovering waku nodes. [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) is a DNS-based discovery protocol that stores [merkle trees](https://en.wikipedia.org/wiki/Merkle_tree) in DNS records which contain connection information for nodes. [EIP 1459: Node Discovery via DNS](https://eips.ethereum.org/EIPS/eip-1459), which is one of the strategies we are using for discovering waku nodes. [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459) is a DNS-based discovery protocol that stores [merkle trees](https://en.wikipedia.org/wiki/Merkle_tree) in DNS records which contain connection information for nodes.
*Waku is our fork of Whisper. Oskar recently wrote an [entire post](https://vac.dev/fixing-whisper-with-waku) explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification [here](https://specs.vac.dev/specs/waku/waku.html)* _Waku is our fork of Whisper. Oskar recently wrote an [entire post](https://vac.dev/fixing-whisper-with-waku) explaining it. In short, Waku is our method of fixing the shortcomings of Whisper in a more iterative fashion. You can find the specification [here](https://specs.vac.dev/specs/waku/waku.html)_
DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be. DNS-based methods for bootstrapping p2p networks are quite popular. Even Bitcoin uses it, but it uses a concept called DNS seeds, which are just DNS servers that are configured to return a list of randomly selected nodes from the network upon being queried. This means that although these seeds are hardcoded in the client, the IP addresses of actual nodes do not have to be.
@ -64,12 +64,13 @@ There are multiple reasons, but the main one is **security**. In the Bitcoin exa
Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party. Let's dive into this. Firstly, a client that is using these DNS records for discovery must know the public key corresponding to the private key controlled by the entity creating the list. This is because the entire list is signed using a secp256k1 private key, giving the client the ability to authenticate the list and know that it has not been tampered with by some external party.
So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using **merkle trees** as follows: So that already makes this a lot safer than the method Bitcoin uses. But how are these lists even stored? As previously stated they are stored using **merkle trees** as follows:
- The root of the tree is stored in a [`TXT` record](https://simpledns.plus/help/txt-records), this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above.
Additionally, there is also a root hash to a second tree called a **link tree**, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names.
The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version. - The root of the tree is stored in a [`TXT` record](https://simpledns.plus/help/txt-records), this record contains the tree's root hash, a sequence number which is incremented every time the tree is updated and a signature as stated above.
Additionally, there is also a root hash to a second tree called a **link tree**, it contains the information to different lists. This link tree allows us to delegate trust and build a graph of multiple merkle trees stored across multiple DNS names.
The sequence number ensures that an attacker cannot replace a tree with an older version because when a client reads the tree, they should ensure that the sequence number is greater than the last synchronized version.
- Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a `TXT` record. The branch record contains all the hashes of the branch's leafs. - Using the root hash for the tree, we can find the merkle tree's first branch, the branch is also stored in a `TXT` record. The branch record contains all the hashes of the branch's leafs.
- Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at [EIP-778](https://eips.ethereum.org/EIPS/eip-778), or read a short blog post I wrote explaining them [here](https://dean.eigenmann.me/blog/2020/01/21/network-addresses-in-ethereum/#enr). - Once a client starts reading all the leafs, they can find one of two things: either a new branch record leading them further down the tree or an Ethereum Name Records (ENR) which means they now have the address of a node to connect to! To learn more about ethereum node records you can have a look at [EIP-778](https://eips.ethereum.org/EIPS/eip-778), or read a short blog post I wrote explaining them [here](https://dean.eigenmann.me/blog/2020/01/21/network-addresses-in-ethereum/#enr).

28
_posts/2020-04-16-wechat-replacement-need.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "What Would a WeChat Replacement Need?" name: 'What Would a WeChat Replacement Need?'
title: "What Would a WeChat Replacement Need?" title: 'What Would a WeChat Replacement Need?'
date: 2020-04-16 12:00:00 +0800 date: 2020-04-16 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /wechat-replacement-need permalink: /wechat-replacement-need
categories: research category: research
summary: What would a self-sovereign, private, censorship-resistant and open alternative to WeChat look like? summary: What would a self-sovereign, private, censorship-resistant and open alternative to WeChat look like?
image: /assets/img/tianstatue.jpg image: /assets/img/tianstatue.jpg
discuss: https://forum.vac.dev/t/discussion-what-would-a-wechat-replacement-need/42 discuss: https://forum.vac.dev/t/discussion-what-would-a-wechat-replacement-need/42
@ -84,11 +84,12 @@ We want something that is self-sovereign, private, censorship-resistant and open
**Open source**. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn't been compromised, e.g. by uploading private data to a third party. **Open source**. Related to above, where we must be able to inspect the software to know that it functions as advertised and hasn't been compromised, e.g. by uploading private data to a third party.
Some of these are graded and a bit subtle, i.e.: Some of these are graded and a bit subtle, i.e.:
- Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible. - Censorship resistance would ideally be able to absorb Internet shutdowns. This would require an extensive MANET/meshnet infrastructure, which while desirable, requires a lot of challenges to be overcome to be feasible.
- Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience. - Privacy would ideally make all actions (optionally) totally anoymous, though this may incur undue costs on bandwidth and latency, which impacts user experience.
- Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals. - Decentralization, certain topologies, such as DHTs, are efficient and quite decentralized but still have some centralized aspects, which makes it attackable in various ways. Ditto for blockchains compared with bearer instruments which requires some coordinating infrastructure, compared with naturally occuring assets such as precious metals.
- "Discover people" and striving for "total anonymity" might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy. - "Discover people" and striving for "total anonymity" might initially seem incompatible. The idea is to provide for sane defaults, and then allow people to decide how much information they want to disclose. This is the essence of privacy.
- Users often want *some* form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement. - Users often want _some_ form of moderation to get a good user experience, which can be seen as a form of censorship. The idea to raise the bar on the basics, the fundamental infrastructure. If individuals or specific communities want certain moderation mechanisms, that is still a compatible requirement.
### Counterpoint 1 ### Counterpoint 1
@ -115,9 +116,10 @@ Starting from the most basic: what is an account and how do you get one? With mo
Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this. Now, we could argue these app providers can do what they want. And they are right, in a very narrow sense. As apps like WeChat (and Google) become general-purpose platforms, they become more and more ingrained in our everyday lives. They start to provide utilities that we absolutely require to work to go about our day, such as paying for food or transportation. This means we need higher standard than this.
Justifications for requiring phone numbers are usually centered around three claims: Justifications for requiring phone numbers are usually centered around three claims:
1) Avoiding spam
2) Tying your account to your real name, for various reasons 1. Avoiding spam
3) Using as a commonly shared identifier as a social network discovery mechanism 2. Tying your account to your real name, for various reasons
3. Using as a commonly shared identifier as a social network discovery mechanism
Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on. Of course, many services require more than phone numbers. E.g. email, other forms of personal data such as voice recording, linking a bank account, and so on.
@ -246,19 +248,11 @@ Corey, Dean, Jacek.
## References ## References
[^1]: Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws [^1]: Mandatory SIM card registration laws: https://privacyinternational.org/long-read/3018/timeline-sim-card-registration-laws
[^2]: On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/ [^2]: On WeChat keyword censorship: https://citizenlab.ca/2016/11/wechat-china-censorship-one-app-two-systems/
[^3]: Net Neutrality: https://www.eff.org/issues/net-neutrality [^3]: Net Neutrality: https://www.eff.org/issues/net-neutrality
[^4]: ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/ [^4]: ISP centralization: https://ilsr.org/repealing-net-neutrality-puts-177-million-americans-at-risk/
[^5]: Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf [^5]: Incentives Build Robustness in BitTorrent bittorrent.org/bittorrentecon.pdf
[^6]: The Case for Electronic Cash: https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf
[^6]: The Case for Electronic Cash: https://coincenter.org/files/2019-02/the-case-for-electronic-cash-coin-center.pdf
[^7]: Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html [^7]: Money, blockchains, and social scalability: http://unenumerated.blogspot.com/2017/02/money-blockchains-and-social-scalability.html
[^8]: Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions [^8]: Zcash private transactions (partial paywall): https://www.theblockcrypto.com/genesis/48413/an-analysis-of-zcashs-private-transactions
[^9]: Shielded transactions usage (stats page 404s): https://z.cash/support/faq/ [^9]: Shielded transactions usage (stats page 404s): https://z.cash/support/faq/

34
_posts/2020-04-27-feasibility-discv5.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "Feasibility Study: Discv5" name: 'Feasibility Study: Discv5'
title: "Feasibility Study: Discv5" title: 'Feasibility Study: Discv5'
date: 2020-04-27 12:00:00 +0800 date: 2020-04-27 12:00:00 +0800
author: dean author: dean
published: true published: true
permalink: /feasibility-discv5 permalink: /feasibility-discv5
categories: research category: research
summary: Looking at discv5 and the theoretical numbers behind finding peers. summary: Looking at discv5 and the theoretical numbers behind finding peers.
discuss: https://discuss.status.im/t/discv5-feasibility-study/1632 discuss: https://discuss.status.im/t/discv5-feasibility-study/1632
--- ---
@ -23,13 +23,13 @@ Namely, we want to ensure censorship resistance to state-level adversaries. One
Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider: Considering we are talking about resource restricted devices, let's look at the implications and what we need to consider:
- **Battery consumption** - constant connections like websockets consume a lot of battery life. - **Battery consumption** - constant connections like websockets consume a lot of battery life.
- **CPU usage** - certain discovery methods may be CPU incentive, slowing an app down and making it unusable. - **CPU usage** - certain discovery methods may be CPU incentive, slowing an app down and making it unusable.
- **Bandwidth consumption** - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans. - **Bandwidth consumption** - a lot of users will be using data plans, the discovery method needs to be efficient in order to accommodate those users without using up significant portions of their data plans.
- **Short connection windows** - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time. - **Short connection windows** - the discovery algorithm needs to be low latency, that means it needs to return results fast. This is because many users will only have the app open for a short amount of time.
- **Not publicly connectable** - There is a good chance that most resource restricted devices are not publicly connectable. - **Not publicly connectable** - There is a good chance that most resource restricted devices are not publicly connectable.
For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node. For a node to be able to participate as both a provider, and a consumer in the discovery method. Meaning a node both reads from other nodes' stored DHTs and hosts the DHT for other nodes to read from, it needs to be publically connectable. This means another node must be able to connect to some public IP of the given node.
With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation. With devices that are behind a NAT, this is easier said than done. Especially mobile devices, that when connected to 4G LTE networks are often stuck behind a symmetric NAT, drastically reducing the the succeess rate of NAT traversal. Keeping this in mind, it becomes obvious that most resource restricted devices will be consumers rather than providers due to this technical limitation.
@ -38,8 +38,9 @@ In order to answer our questions, we formulated the problem with a simple method
## Overview ## Overview
The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer: The main things we wanted to investigate was the overhead on finding a peer. This means we wanted to look at both the bandwidth, latency and effectiveness of this. There are 2 methods which we can use to find a peer:
- We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.
- We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through [topic tables](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement). - We can find a peer with a specific ID, using normal lookup methods as documented by Kademlia.
- We can find a peer that advertises a capability, this is possible using either capabilities advertised in the ENR or through [topic tables](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#topic-advertisement).
## Feasbility ## Feasbility
@ -76,7 +77,8 @@ Research has shown that finding a node in the DHT has a relatively low effect on
One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more "features" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes. One of the main blockers right now is figuring out what the CPU and memory usage of discv5 is on mobile phones, this is a large blocker as it affects one of the core problems for us. We need to consider whether discv5 is an upgrade as it allows us to simplify our current discovery process or if it is too much of an overhead for resource restricted devices. The topic table feature could largely enhance discovery however it is not yet implemented. Given that CPU and memory isn't too high, discv5 could probably be used as the other issues are more "features" than large scale issues. Implementing it would already reduce the ability for state level adversaries to censor our nodes.
## Acknowledgements ## Acknowledgements
- Oskar Thoren
- Dmitry Shmatko - Oskar Thoren
- Kim De Mey - Dmitry Shmatko
- Corey Petty - Kim De Mey
- Corey Petty

23
_posts/2020-04-9-kademlia-to-discv5.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "From Kademlia to Discv5" name: 'From Kademlia to Discv5'
title: "From Kademlia to Discv5" title: 'From Kademlia to Discv5'
date: 2020-04-9 16:00:00 +0100 date: 2020-04-9 16:00:00 +0100
author: dean author: dean
published: true published: true
permalink: /kademlia-to-discv5 permalink: /kademlia-to-discv5
categories: research category: research
summary: A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ. summary: A quick history of discovery in peer-to-peer networks, along with a look into discv4 and discv5, detailing what they are, how they work and where they differ.
--- ---
@ -48,9 +48,10 @@ XOR 10011001
The top in decimal numbers means that the distance between `153` and `50` is `171`. The top in decimal numbers means that the distance between `153` and `50` is `171`.
There are several reasons why `XOR` was taken: There are several reasons why `XOR` was taken:
1. The distance from one ID to itself will be `0`.
2. Distance is symmetric, A to B is the same as B to A. 1. The distance from one ID to itself will be `0`.
3. Follows triangle inequality, if `A`, `B` and `C` are points on a triangle then the distance `A` to `B` is closer or equal to that of `A` to `C` plus the one from `B` to `C`. 2. Distance is symmetric, A to B is the same as B to A.
3. Follows triangle inequality, if `A`, `B` and `C` are points on a triangle then the distance `A` to `B` is closer or equal to that of `A` to `C` plus the one from `B` to `C`.
In summary, this distance function allows a node to decide what is "close" to it and make decisions based on that "closeness". In summary, this distance function allows a node to decide what is "close" to it and make decisions based on that "closeness".
@ -67,11 +68,11 @@ For Kademlia nodes to support these functions, there are several messages with w
- `FINDNODE` - Returns the closest nodes requested to a given ID. - `FINDNODE` - Returns the closest nodes requested to a given ID.
- `FINDVALUE` - The same as `FINDNODE`, except if a node stores the specific value it will return it directly. - `FINDVALUE` - The same as `FINDNODE`, except if a node stores the specific value it will return it directly.
*This is a **very** simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the [paper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) or a more in-depth [design specification](http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html)* _This is a **very** simplified explanation of Kademlia and skips various important details. For the full description, make sure to check out the [paper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) or a more in-depth [design specification](http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html)_
## Discv4 ## Discv4
Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT. Now after that history lesson, we finally get to discv4 (which stands for discovery v4), Ethereum's current node discovery protocol. The protocol itself is essentially based off of Kademlia, however it does away with certain aspects of it. For example, it does away with any usage of the value part of the DHT.
Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the `FINDVALUE` and `STORE` commands described by Kademlia. Kademlia is mainly used for the organisation of the network, so we only use the routing table to locate other nodes. Due to the fact that discv4 doesn't use the value portion of the DHT at all, we can throw away the `FINDVALUE` and `STORE` commands described by Kademlia.
@ -81,7 +82,7 @@ Additionally, discv4 adds mutual endpoint verification. This is meant to ensure
Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called `ENRRequest`. Finally, all discv4 nodes are expected to maintain up-to-date ENR records. These contain information about a node. They can be requested from any node using a discv4-specific packet called `ENRRequest`.
*If you want some more details on ENRs, check out one of my posts ["Network Addresses in Ethereum"](https://dean.eigenmann.me/blog/2020/01/21/network-addresses-in-ethereum/)* _If you want some more details on ENRs, check out one of my posts ["Network Addresses in Ethereum"](https://dean.eigenmann.me/blog/2020/01/21/network-addresses-in-ethereum/)_
Discv4 comes with its own range of problems however. Let's look at a few of them. Discv4 comes with its own range of problems however. Let's look at a few of them.
@ -103,7 +104,7 @@ Logarithmic distance means we first calculate the distance and then run it throu
log2(A xor B) log2(A xor B)
``` ```
And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are [first in first out](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)) lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering `ads` on their peers. And the second, more important change, is that discv5 aims at solving one of the biggest issues of discv4: the differentiation of sub-protocols. It does this by adding topic tables. Topic tables are [first in first out](<https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)>) lists that contain nodes which have advertised that they provide a specific service. Nodes get themselves added to this list by registering `ads` on their peers.
As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place `ads` on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network. As of writing, there is still an issue with this proposal. There is currently no efficient way for a node to place `ads` on multiple peers, since it would require separate requests for every peer which is inefficient in a large-scale network.

12
_posts/2020-07-01-problem-4.md
View File

@ -1,13 +1,13 @@
--- ---
layout: post layout: post
name: "Problem 4 Wallet stacking sats block height" name: 'Problem 4 Wallet stacking sats block height'
title: "Problem 4 Wallet stacking sats block height" title: 'Problem 4 Wallet stacking sats block height'
date: 2020-07-01 12:00:00 +0800 date: 2020-07-01 12:00:00 +0800
author: franck author: franck
published: true published: true
permalink: /status-ipsum-4 permalink: /status-ipsum-4
category: problem category: problem
summary: "Status ipsum lorem exequoe" summary: 'Status ipsum lorem exequoe'
image: /assets/img/js-waku-gist.png image: /assets/img/js-waku-gist.png
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82 discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82
--- ---
@ -117,7 +117,7 @@ we were able to fine tune the API to provide a simple and succinct interface.
You can start a node, connect to other nodes and send a message in less than ten lines of code: You can start a node, connect to other nodes and send a message in less than ten lines of code:
```javascript ```javascript
import { Waku } from "js-waku"; import { Waku } from 'js-waku';
const waku = await Waku.create({}); const waku = await Waku.create({});
@ -125,8 +125,8 @@ const nodes = await getStatusFleetNodes();
await Promise.all(nodes.map((addr) => waku.dial(addr))); await Promise.all(nodes.map((addr) => waku.dial(addr)));
const msg = WakuMessage.fromUtf8String( const msg = WakuMessage.fromUtf8String(
"Here is a message!", 'Here is a message!',
"/my-cool-app/1/my-use-case/proto" '/my-cool-app/1/my-use-case/proto'
); );
await waku.relay.send(msg); await waku.relay.send(msg);
``` ```

46
_posts/2020-07-01-waku-v2-pitch.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "What's the Plan for Waku v2?" name: "What's the Plan for Waku v2?"
title: "What's the Plan for Waku v2?" title: "What's the Plan for Waku v2?"
date: 2020-07-01 12:00:00 +0800 date: 2020-07-01 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /waku-v2-plan permalink: /waku-v2-plan
categories: research category: research
summary: Read about our plans for Waku v2, moving to libp2p, better routing, adaptive nodes and accounting! summary: Read about our plans for Waku v2, moving to libp2p, better routing, adaptive nodes and accounting!
image: /assets/img/status_scaling_model_fig4.png image: /assets/img/status_scaling_model_fig4.png
discuss: https://forum.vac.dev/t/waku-version-2-pitch/52 discuss: https://forum.vac.dev/t/waku-version-2-pitch/52
@ -14,7 +14,7 @@ discuss: https://forum.vac.dev/t/waku-version-2-pitch/52
**tldr: The Waku network is fragile and doesn't scale. Here's how to solve it.** **tldr: The Waku network is fragile and doesn't scale. Here's how to solve it.**
*NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.* _NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments._
# Problem # Problem
@ -26,12 +26,12 @@ Based on user acquisition models, the initial goal is to support 100k DAU in Sep
With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions. With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.
****1. Connection limits****. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2. \***\*1. Connection limits\*\***. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.
****2. Bandwidth as a bottleneck****. We notice that memory usage appears to not be \***\*2. Bandwidth as a bottleneck\*\***. We notice that memory usage appears to not be
the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5. the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.
****3. Amplification factors****. Reducing amplification factors with better routing, would have a high impact, but it is likely we'd need additional measures as well, such as topic sharding or similar. See Fig 8-13. \***\*3. Amplification factors\*\***. Reducing amplification factors with better routing, would have a high impact, but it is likely we'd need additional measures as well, such as topic sharding or similar. See Fig 8-13.
Figure 1-5: Figure 1-5:
@ -50,12 +50,12 @@ What we need to do is:
Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we: Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:
- keep increasing requirements for Internet speed for full nodes - keep increasing requirements for Internet speed for full nodes
- are vulnerable to censorship and attacks - are vulnerable to censorship and attacks
- have to control the topology in an artifical manner to keep up with load - have to control the topology in an artifical manner to keep up with load
- basically re-invent a traditional centralized client-server app with extra steps - basically re-invent a traditional centralized client-server app with extra steps
- deliberately ignore most of our principles - deliberately ignore most of our principles
- risk the network being shut down when we run out of cash - risk the network being shut down when we run out of cash
# Appetite # Appetite
@ -79,7 +79,6 @@ Let's first look at the baseline, and then go into some of the tracks and their
Here's where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can't easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details. Here's where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can't easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.
![](assets/img/waku_v1_routing_small.png) ![](assets/img/waku_v1_routing_small.png)
## Track 1 - Move to libp2p ## Track 1 - Move to libp2p
@ -100,7 +99,6 @@ Why can't we use Waku topics for routing directly? PubSub over libp2p isn't buil
![](assets/img/waku_v2_routing_flood_small.png) ![](assets/img/waku_v2_routing_flood_small.png)
Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below. Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.
Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: <https://github.com/status-im/nim-waku> Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: <https://github.com/status-im/nim-waku>
@ -201,9 +199,9 @@ The general shape of the solution is inspired by the Swarm model, where we do ac
While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as: While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:
- end to end delivery - end to end delivery
- online time - online time
- completeness of storage - completeness of storage
This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See <https://discuss.status.im/t/network-incentivisation-first-draft/1037> for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And <https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51> for using validity proofs and removing liveness requirement for settlement. This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See <https://discuss.status.im/t/network-incentivisation-first-draft/1037> for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And <https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51> for using validity proofs and removing liveness requirement for settlement.
@ -227,14 +225,14 @@ In accounting phase it is largely assumed nodes are honest. What happens when th
For light nodes: For light nodes:
- if they don't, they get disconnected - if they don't, they get disconnected
- prepayment (especially to "high value" nodes) - prepayment (especially to "high value" nodes)
For full nodes: For full nodes:
- multiple nodes reporting to agree, where truth becomes a shelling point - multiple nodes reporting to agree, where truth becomes a shelling point
- use eigentrust - use eigentrust
- staking for discovery visibility with slashing - staking for discovery visibility with slashing
### 5. Settlement PoC ### 5. Settlement PoC

12
_posts/2020-09-28-problem-2.md
View File

@ -1,13 +1,13 @@
--- ---
layout: post layout: post
name: "Problem No. 2, very long title Bitcoin ipsum dolor sit amet. Peer-to-peer whitepaper SHA-256 bitcoin." name: 'Problem No. 2, very long title Bitcoin ipsum dolor sit amet. Peer-to-peer whitepaper SHA-256 bitcoin.'
title: "Problem No. 2, very long title Bitcoin ipsum dolor sit amet. Peer-to-peer whitepaper SHA-256 bitcoin." title: 'Problem No. 2, very long title Bitcoin ipsum dolor sit amet. Peer-to-peer whitepaper SHA-256 bitcoin.'
date: 2020-09-28 12:00:00 +0800 date: 2020-09-28 12:00:00 +0800
author: franck author: franck
published: true published: true
permalink: /status-ipsum-2 permalink: /status-ipsum-2
category: problem category: problem
summary: "Status ipsum lorem exequoe" summary: 'Status ipsum lorem exequoe'
image: /assets/img/js-waku-gist.png image: /assets/img/js-waku-gist.png
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82 discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82
--- ---
@ -117,7 +117,7 @@ we were able to fine tune the API to provide a simple and succinct interface.
You can start a node, connect to other nodes and send a message in less than ten lines of code: You can start a node, connect to other nodes and send a message in less than ten lines of code:
```javascript ```javascript
import { Waku } from "js-waku"; import { Waku } from 'js-waku';
const waku = await Waku.create({}); const waku = await Waku.create({});
@ -125,8 +125,8 @@ const nodes = await getStatusFleetNodes();
await Promise.all(nodes.map((addr) => waku.dial(addr))); await Promise.all(nodes.map((addr) => waku.dial(addr)));
const msg = WakuMessage.fromUtf8String( const msg = WakuMessage.fromUtf8String(
"Here is a message!", 'Here is a message!',
"/my-cool-app/1/my-use-case/proto" '/my-cool-app/1/my-use-case/proto'
); );
await waku.relay.send(msg); await waku.relay.send(msg);
``` ```

32
_posts/2020-09-28-waku-v2-update.md
View File

@ -1,12 +1,12 @@
--- ---
layout: post layout: post
name: "Waku v2 Update" name: 'Waku v2 Update'
title: "Waku v2 Update" title: 'Waku v2 Update'
date: 2020-09-28 12:00:00 +0800 date: 2020-09-28 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /waku-v2-update permalink: /waku-v2-update
categories: research category: research
summary: A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next? summary: A research log. Read on to find out what is going on with Waku v2, a messaging protocol. What has been happening? What is coming up next?
image: /assets/img/vac.png image: /assets/img/vac.png
discuss: https://forum.vac.dev/t/discussion-waku-v2-update/56 discuss: https://forum.vac.dev/t/discussion-waku-v2-update/56
@ -72,16 +72,16 @@ Now that we know what the current state is, what is still missing? what are the
While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope: While we are getting closer to closing out work for track 1, there are still a few things missing from the initial scope:
1) Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level. 1. Store and filter protocols need to be finished. This means basic spec, implementation, API integration and proven to work in a testnet. All of these are work in progress and expected to be done very soon. Once the store protocol is done in a basic form, it needs further improvements to make it production ready, at least on a spec/basic implementation level.
2) Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status ([status-nim](github.com/status-im/status-nim)) has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings 2. Core integration was mentioned in scope for track 1 initially. This work has stalled a bit, largely due to organizational bandwidth and priorities. While there is a Nim Node API that in theory is ready to be used, having it be used in e.g. Status desktop or mobile app is a different matter. The team responsible for this at Status ([status-nim](github.com/status-im/status-nim)) has been making progress on getting nim-waku v1 integrated, and is expected to look into nim-waku v2 integration soon. One thing that makes this a especially tricky is the difference in interface between Waku v1 and v2, which brings
us too... us too...
3) Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 ([1](https://github.com/vacp2p/specs/issues/158), [2](https://github.com/vacp2p/specs/issues/181)). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc). 3. Companion spec for encryption. As part of simplifying the protocol, the routing is decoupled from the encryption in v2 ([1](https://github.com/vacp2p/specs/issues/158), [2](https://github.com/vacp2p/specs/issues/181)). There are multiple layers of encryption at play here, and we need to figure out a design that makes sense for various use cases (dapps using Waku on their own, Status app, etc).
4) Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented. 4. Bridge implementation. The spec is done and we know how it should work, but it needs to be implemented.
5) General tightening up of specs and implementation. 5. General tightening up of specs and implementation.
While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope. While this might seem like a lot, a lot has been done already, and the majority of the remaining tasks are more amendable to be pursued in parallel with other efforts. It is also worth mentioning that part of track 2 and 3 have been started, in the form of moving to GossipSub (amplification factors) and basics of adaptive nodes (multiple protocols). This is in addition to things like Waku Web which were not part of the initial scope.
@ -89,16 +89,16 @@ While this might seem like a lot, a lot has been done already, and the majority
Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details. Aside from the things mentioned above, what is coming up next? There are a few areas of interest, mentioned in no particular order. For track 2 and 3, see previous post for more details.
1) Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors. 1. Better routing (track 2). While we are already building on top of GossipSub, we still need to explore things like topic sharding in more detail to further reduce amplification factors.
2) Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner. 2. Accounting and user-run nodes (track 3). With store and filter protocol getting ready, we can start to implement accounting and light connection game for incentivization in a bottom up and iterative manner.
3) Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet 3. Privacy research. Study better and more rigorous privacy guarantees. E.g. how FloodSub/GossipSub behaves for common threat models, and how custom packet
format can improve things like unlinkability. format can improve things like unlinkability.
4) zkSnarks RLN for spam protection and incentivization. We studied this [last year](https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks) and recent developments have made this relevant to study again. Create an [experimental spec/PoC](https://github.com/vacp2p/specs/issues/189) as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this! 4. zkSnarks RLN for spam protection and incentivization. We studied this [last year](https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks) and recent developments have made this relevant to study again. Create an [experimental spec/PoC](https://github.com/vacp2p/specs/issues/189) as an extension to the relay protocol. Kudos to Barry Whitehat and others like Kobi Gurkan and Koh Wei Jie for pushing this!
5) Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we'd like to deliver on that. 5. Ethereum M2M messaging. Being able to run in the browser opens up a lot of doors, and there is an opportunity here to enable things like a decentralized WalletConnect, multi-sig transactions, voting and similar use cases. This was the original goal of Whisper, and we'd like to deliver on that.
As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates. As you can tell, quite a lot of thing! Luckily, we have two people joining as protocol engineers soon, which will bring much needed support for the current team of ~2-2.5 people. More details to come in further updates.

6
_posts/2020-11-10-problem-3.md
View File

@ -1,13 +1,13 @@
--- ---
layout: post layout: post
name: "Problem No. 3 Inputs cryptocurrency key pair timestamp server, difficulty outputs! Key pair, halvening consensus " name: 'Problem No. 3 Inputs cryptocurrency key pair timestamp server, difficulty outputs! Key pair, halvening consensus '
title: "Problem No. 3 Inputs cryptocurrency key pair timestamp server, difficulty outputs! Key pair, halvening consensus " title: 'Problem No. 3 Inputs cryptocurrency key pair timestamp server, difficulty outputs! Key pair, halvening consensus '
date: 2020-11-10 12:00:00 +0800 date: 2020-11-10 12:00:00 +0800
author: franck author: franck
published: true published: true
permalink: /status-ipsum-3 permalink: /status-ipsum-3
category: problem category: problem
summary: "Status ipsum lorem exequoe" summary: 'Status ipsum lorem exequoe'
image: /assets/img/js-waku-gist.png image: /assets/img/js-waku-gist.png
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82 discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82
--- ---

58
_posts/2020-11-10-waku-v2-ethereum-messaging.md
View File

@ -1,18 +1,18 @@
--- ---
layout: post layout: post
name: "[Talk] Vac, Waku v2 and Ethereum Messaging" name: '[Talk] Vac, Waku v2 and Ethereum Messaging'
title: "[Talk] Vac, Waku v2 and Ethereum Messaging" title: '[Talk] Vac, Waku v2 and Ethereum Messaging'
date: 2020-11-10 12:00:00 +0800 date: 2020-11-10 12:00:00 +0800
author: oskarth author: oskarth
published: true published: true
permalink: /waku-v2-ethereum-messaging permalink: /waku-v2-ethereum-messaging
categories: research category: research
summary: Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging. summary: Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging.
image: /assets/img/taipei_ethereum_meetup_slide.png image: /assets/img/taipei_ethereum_meetup_slide.png
discuss: https://forum.vac.dev/t/discussion-talk-vac-waku-v2-and-ethereum-messaging/60 discuss: https://forum.vac.dev/t/discussion-talk-vac-waku-v2-and-ethereum-messaging/60
--- ---
*The following post is a transcript of the talk given at the [Taipei Ethereum meetup, November 5](https://www.meetup.com/Taipei-Ethereum-Meetup/events/274033344/). There is also a [video recording]( https://www.youtube.com/watch?v=lUDy1MoeYnI).* _The following post is a transcript of the talk given at the [Taipei Ethereum meetup, November 5](https://www.meetup.com/Taipei-Ethereum-Meetup/events/274033344/). There is also a [video recording](https://www.youtube.com/watch?v=lUDy1MoeYnI)._
--- ---
@ -68,9 +68,9 @@ Taking a step back, what problem does Waku v2 attempt to solve compared to all t
3. **Resource restricted**. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.: 3. **Resource restricted**. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:
- limited bandwidth, CPU, memory, disk, battery, etc - limited bandwidth, CPU, memory, disk, battery, etc
- not being publicly connectable - not being publicly connectable
- only being intermittently connected; mostly-offline - only being intermittently connected; mostly-offline
4. **Privacy**. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc. 4. **Privacy**. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.
@ -86,12 +86,12 @@ It is running over libp2p and we had our second internal testnet last week or so
The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2: The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:
- Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices. - Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.
- Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well. - Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.
- Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline. - Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.
- Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about. - Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.
- Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1. - Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.
- Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility. - Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.
Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production. Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.
@ -109,10 +109,10 @@ There are also some changes we made to at what level content encryption happens,
More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like: More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:
- Better scaling, by using topic sharding. - Better scaling, by using topic sharding.
- Accounting and user-run nodes, to account for and incentives full nodes. - Accounting and user-run nodes, to account for and incentives full nodes.
- Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc. - Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.
- Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before. - Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.
As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next. As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next.
@ -126,12 +126,12 @@ It is easy to think that Waku v2 is only for human to human messaging, since tha
What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as: What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:
- Reducing on-chain transactions - Reducing on-chain transactions
- Reduce latency for operations - Reduce latency for operations
- Decentralize centrally coordinated services (like WalletConnect) - Decentralize centrally coordinated services (like WalletConnect)
- Improve UX of dapps - Improve UX of dapps
- Broadcast live information - Broadcast live information
- A message transport layer for state channels - A message transport layer for state channels
And so on. And so on.
@ -139,11 +139,11 @@ And so on.
What are some examples of practical things Waku as used for Ethereum Messaging could solve? What are some examples of practical things Waku as used for Ethereum Messaging could solve?
- Multisig transfers only needing one on chain transaction - Multisig transfers only needing one on chain transaction
- DAO votes only needing one one chain transaction - DAO votes only needing one one chain transaction
- Giving dapps ability to direct push notifications to users - Giving dapps ability to direct push notifications to users
- Giving users ability to directly respond to requests from daps - Giving users ability to directly respond to requests from daps
- Decentralized Wallet Connect - Decentralized Wallet Connect
Etc. Etc.

2
_posts/2021-03-03-rln-relay.md
View File

@ -6,7 +6,7 @@ date: 2021-03-05 12:00:00 +0800
author: sanaztaheri author: sanaztaheri
published: true published: true
permalink: /rln-relay permalink: /rln-relay
categories: research category: research
summary: This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end. summary: This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.
image: /assets/img/rain.png image: /assets/img/rain.png
discuss: https://forum.vac.dev/t/privacy-preserving-p2p-economic-spam-protection-in-waku-v2-with-rate-limiting-nullfiers/66 discuss: https://forum.vac.dev/t/privacy-preserving-p2p-economic-spam-protection-in-waku-v2-with-rate-limiting-nullfiers/66

4
_posts/2021-03-05-problem-1.md
View File

@ -1,7 +1,7 @@
--- ---
layout: post layout: post
name: "Problem No. 1 - Status ipsum lorem exequoe" name: 'Problem No. 1 - Status ipsum lorem exequoe'
title: "Problem No. 1 - Status ipsum lorem exequoe" title: 'Problem No. 1 - Status ipsum lorem exequoe'
date: 2021-03-05 12:00:00 +0800 date: 2021-03-05 12:00:00 +0800
author: franck author: franck
published: true published: true

19
_posts/2021-06-04-presenting-js-waku.md
View File

@ -1,15 +1,15 @@
--- ---
layout: post layout: post
name: "Presenting JS-Waku: Waku v2 in the Browser" name: 'Presenting JS-Waku: Waku v2 in the Browser'
title: "Presenting JS-Waku: Waku v2 in the Browser" title: 'Presenting JS-Waku: Waku v2 in the Browser'
date: 2021-06-04 12:00:00 +0800 date: 2021-06-04 12:00:00 +0800
author: franck author: franck
published: true published: true
permalink: /presenting-js-waku permalink: /presenting-js-waku
categories: platform category: research
summary: "JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!" summary: 'JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!'
image: /assets/img/js-waku-gist.png image: /assets/img/js-waku-gist.png
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81 discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81
--- ---
For the past 3 months, we have been working on bringing Waku v2 to the browser. For the past 3 months, we have been working on bringing Waku v2 to the browser.
@ -61,7 +61,7 @@ It showed how using js-libp2p with few modifications enabled access to the Waku
There was still some unresolved challenges. There was still some unresolved challenges.
For example, nim-waku only support TCP connections which are not supported by browser applications. For example, nim-waku only support TCP connections which are not supported by browser applications.
Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets, Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,
which in turn could connect to wakunode2 via TCP. which in turn could connect to wakunode2 via TCP.
However, to enable dApp and Wallet developers to easily integrate Waku in their product, However, to enable dApp and Wallet developers to easily integrate Waku in their product,
we need to give them a library that is easy to use and works out of the box: we need to give them a library that is easy to use and works out of the box:
@ -109,7 +109,7 @@ and fix small nim-waku & nim-libp2p bugs
To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes. To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.
A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser. A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.
Unfortunately websocket support is not yet available in [nim-libp2p](https://github.com/status-im/nim-libp2p/issues/407) so Unfortunately websocket support is not yet available in [nim-libp2p](https://github.com/status-im/nim-libp2p/issues/407) so
we ended up deploying [websockify](https://github.com/novnc/websockify) alongside wakunode2 instances. we ended up deploying [websockify](https://github.com/novnc/websockify) alongside wakunode2 instances.
As we built the [web chat app](https://github.com/status-im/js-waku/tree/main/examples/web-chat), As we built the [web chat app](https://github.com/status-im/js-waku/tree/main/examples/web-chat),
@ -124,7 +124,10 @@ const waku = await Waku.create({});
const nodes = await getStatusFleetNodes(); const nodes = await getStatusFleetNodes();
await Promise.all(nodes.map((addr) => waku.dial(addr))); await Promise.all(nodes.map((addr) => waku.dial(addr)));
const msg = WakuMessage.fromUtf8String("Here is a message!", "/my-cool-app/1/my-use-case/proto") const msg = WakuMessage.fromUtf8String(
'Here is a message!',
'/my-cool-app/1/my-use-case/proto'
);
await waku.relay.send(msg); await waku.relay.send(msg);
``` ```