Create post layout
This commit is contained in:
parent
1db17ebcc9
commit
7ba7f7d1a7
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
layout: default
|
||||
---
|
||||
|
||||
<section
|
||||
class="container max-w-screen-xl flex flex-col sm:flex-row pt-10 border-b"
|
||||
>
|
||||
<div
|
||||
class="
|
||||
heading-block
|
||||
w-full
|
||||
sm:w-2/12
|
||||
lg:w-3/12
|
||||
flex
|
||||
lg:justify-center
|
||||
items-start
|
||||
pb-3
|
||||
sm:pb-0
|
||||
"
|
||||
>
|
||||
<a
|
||||
class="link link--back"
|
||||
href="{{site.url}}{{ site.baseurl }}/research-log/"
|
||||
>Back</a
|
||||
>
|
||||
</div>
|
||||
<div class="info-block w-full sm:w-10/12 lg:w-9/12">
|
||||
<div class="post mb-10">
|
||||
<h1 class="text-xl md:text-xxl mb-5 sm:max-w-md lg:max-w-2xl">
|
||||
{{page.title}}
|
||||
</h1>
|
||||
<div>
|
||||
<span class="text-s lg:text-base">
|
||||
{{ page.date | date_to_string }} • by
|
||||
</span>
|
||||
<a
|
||||
href="/authors/{{ page.author }}"
|
||||
class="text-s lg:text-base font-bold"
|
||||
>{{ page.author }}</a
|
||||
>
|
||||
</div>
|
||||
</div>
|
||||
<div class="post__content">{{ content }}</div>
|
||||
</div>
|
||||
</section>
|
|
@ -12,167 +12,98 @@ image: /assets/img/js-waku-gist.png
|
|||
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82
|
||||
---
|
||||
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the browser.
|
||||
Our aim is to empower dApps with Waku v2, and it led to the creation of a new library.
|
||||
We believe now is good time to introduce it!
|
||||
<p class="post__intro">
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the
|
||||
browser. Our aim is to empower dApps with Waku v2, and it led to the creation
|
||||
of a new library. We believe now is good time to introduce it!
|
||||
</p>
|
||||
|
||||
## Waku v2
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2</h2>
|
||||
<p class="post__text">
|
||||
First, let’s review what Waku v2 is and what problem it is trying to solve.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution
|
||||
for the Status app to achieve decentralised communications on resource
|
||||
restricted devices (i.e., mobile phones). The Status chat feature was
|
||||
initially built over Whisper. However, Whisper has a number of caveats which
|
||||
makes it inefficient for mobile phones. For example, with Whisper, all
|
||||
devices are receiving all messages which is not ideal for limited data
|
||||
plans.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was
|
||||
introduced. To further enable web and restricted resource environments, Waku
|
||||
v2 was created based on libp2p. The migration of the Status chat feature to
|
||||
Waku v2 is currently in progress. We see the need of such solution in the
|
||||
broader Ethereum ecosystem, beyond Status.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
This is why we are building Waku v2 as a decentralised communication
|
||||
platform for all to use and build on. If you want to read more about Waku v2
|
||||
and what it aims to achieve, checkout What’s the Plan for Waku v2?.Since
|
||||
last year, we have been busy defining and implementing Waku v2 protocols in
|
||||
nim-waku, from which you can build wakunode2. Wakunode2 is an adaptive and
|
||||
modular Waku v2 node, it allows users to run their own node and use the Waku
|
||||
v2 protocols they need. The nim-waku project doubles as a library, that can
|
||||
be used to add Waku v2 support to native applications.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
First, let's review what Waku v2 is and what problem it is trying to solve.
|
||||
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised
|
||||
communications on resource restricted devices (i.e., mobile phones).
|
||||
|
||||
The Status chat feature was initially built over Whisper.
|
||||
However, Whisper has a number of caveats which makes it inefficient for mobile phones.
|
||||
For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.
|
||||
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced.
|
||||
To further enable web and restricted resource environments, Waku v2 was created based on libp2p.
|
||||
The migration of the Status chat feature to Waku v2 is currently in progress.
|
||||
|
||||
We see the need of such solution in the broader Ethereum ecosystem, beyond Status.
|
||||
This is why we are building Waku v2 as a decentralised communication platform for all to use and build on.
|
||||
If you want to read more about Waku v2 and what it aims to achieve,
|
||||
checkout [What's the Plan for Waku v2?](/waku-v2-plan).
|
||||
|
||||
Since last year, we have been busy defining and implementing Waku v2 protocols in [nim-waku](https://github.com/status-im/nim-waku),
|
||||
from which you can build [wakunode2](https://github.com/status-im/nim-waku#wakunode).
|
||||
Wakunode2 is an adaptive and modular Waku v2 node,
|
||||
it allows users to run their own node and use the Waku v2 protocols they need.
|
||||
The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.
|
||||
|
||||
## Waku v2 in the browser
|
||||
|
||||
We believe that dApps and wallets can benefit from the Waku network in several ways.
|
||||
For some dApps, it makes sense to enable peer-to-peer communications.
|
||||
For others, machine-to-machine communications would be a great asset.
|
||||
For example, in the case of a DAO,
|
||||
Waku could be used for gas-less voting.
|
||||
Enabling the DAO to notify their users of a new vote,
|
||||
and users to vote without interacting with the blockchain and spending gas.
|
||||
|
||||
[Murmur](https://github.com/status-im/murmur) was the first attempt to bring Whisper to the browser,
|
||||
acting as a bridge between devp2p and libp2p.
|
||||
Once Waku v2 was started and there was a native implementation on top of libp2p,
|
||||
a [chat POC](https://github.com/vacp2p/waku-web-chat) was created to demonstrate the potential of Waku v2
|
||||
in web environment.
|
||||
It showed how using js-libp2p with few modifications enabled access to the Waku v2 network.
|
||||
There was still some unresolved challenges.
|
||||
For example, nim-waku only support TCP connections which are not supported by browser applications.
|
||||
Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,
|
||||
which in turn could connect to wakunode2 via TCP.
|
||||
|
||||
However, to enable dApp and Wallet developers to easily integrate Waku in their product,
|
||||
we need to give them a library that is easy to use and works out of the box:
|
||||
introducing [JS-Waku](https://github.com/status-im/js-waku).
|
||||
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network.
|
||||
It is available right now on [npm](https://www.npmjs.com/package/js-waku):
|
||||
|
||||
`npm install js-waku`.
|
||||
|
||||
As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.
|
||||
|
||||
Key Waku v2 protocols are already available:
|
||||
[message](https://rfc.vac.dev/spec/14/), [store](https://rfc.vac.dev/spec/13/), [relay](https://rfc.vac.dev/spec/11/) and [light push](https://rfc.vac.dev/spec/19/),
|
||||
enabling your dApp to:
|
||||
|
||||
- Send and receive near-instant messages on the Waku network (relay),
|
||||
- Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
|
||||
- Send messages with confirmations (light push).
|
||||
|
||||
JS-Waku needs to operate in the same context from which Waku v2 was born:
|
||||
a restricted environment were connectivity or uptime are not guaranteed;
|
||||
JS-Waku brings Waku v2 to the browser.
|
||||
|
||||
## Achievements so far
|
||||
|
||||
We focused the past month on developing a [ReactJS Chat App](https://status-im.github.io/js-waku/).
|
||||
The aim was to create enough building blocks in JS-Waku to enable this showcase web app that
|
||||
we now [use for dogfooding](https://github.com/status-im/nim-waku/issues/399) purposes.
|
||||
|
||||
Most of the effort was on getting familiar with the [js-libp2p](https://github.com/libp2p/js-libp2p) library
|
||||
that we heavily rely on.
|
||||
JS-Waku is the second implementation of Waku v2 protocol,
|
||||
so a lot of effort on interoperability was needed.
|
||||
For example, to ensure compatibility with the nim-waku reference implementation,
|
||||
we run our [tests against wakunode2](https://github.com/status-im/js-waku/blob/90c90dea11dfd1277f530cf5d683fb92992fe141/src/lib/waku_relay/index.spec.ts#L137) as part of the CI.
|
||||
|
||||
This interoperability effort helped solidify the current Waku v2 specifications:
|
||||
By clarifying the usage of topics
|
||||
([#327](https://github.com/vacp2p/rfc/issues/327), [#383](https://github.com/vacp2p/rfc/pull/383)),
|
||||
fix discrepancies between specs and nim-waku
|
||||
([#418](https://github.com/status-im/nim-waku/issues/418), [#419](https://github.com/status-im/nim-waku/issues/419))
|
||||
and fix small nim-waku & nim-libp2p bugs
|
||||
([#411](https://github.com/status-im/nim-waku/issues/411), [#439](https://github.com/status-im/nim-waku/issues/439)).
|
||||
|
||||
To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.
|
||||
A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.
|
||||
Unfortunately websocket support is not yet available in [nim-libp2p](https://github.com/status-im/nim-libp2p/issues/407) so
|
||||
we ended up deploying [websockify](https://github.com/novnc/websockify) alongside wakunode2 instances.
|
||||
|
||||
As we built the [web chat app](https://github.com/status-im/js-waku/tree/main/examples/web-chat),
|
||||
we were able to fine tune the API to provide a simple and succinct interface.
|
||||
You can start a node, connect to other nodes and send a message in less than ten lines of code:
|
||||
|
||||
```javascript
|
||||
import { Waku } from "js-waku";
|
||||
|
||||
const waku = await Waku.create({});
|
||||
|
||||
const nodes = await getStatusFleetNodes();
|
||||
await Promise.all(nodes.map((addr) => waku.dial(addr)));
|
||||
|
||||
const msg = WakuMessage.fromUtf8String(
|
||||
"Here is a message!",
|
||||
"/my-cool-app/1/my-use-case/proto"
|
||||
);
|
||||
await waku.relay.send(msg);
|
||||
```
|
||||
|
||||
We have also put a bounty at [0xHack](https://0xhack.dev/) for using JS-Waku
|
||||
and running a [workshop](https://vimeo.com/551509621).
|
||||
We were thrilled to have a couple of hackers create new software using our libraries.
|
||||
One of the projects aimed to create a decentralised, end-to-end encrypted messenger app,
|
||||
similar to what the [ETH-DM](https://rfc.vac.dev/spec/20/) protocol aims to achieve.
|
||||
Another project was a decentralised Twitter platform.
|
||||
Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.
|
||||
|
||||
As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network
|
||||
while preserving this out of the box experience.
|
||||
|
||||
## What's next?
|
||||
|
||||
Next, we are directing our attention towards [Developer Experience](https://github.com/status-im/js-waku/issues/68).
|
||||
We already have [documentation](https://www.npmjs.com/package/js-waku) available but we want to provide more:
|
||||
[Tutorials](https://github.com/status-im/js-waku/issues/56), various examples
|
||||
and showing how [JS-Waku can be used with Web3](https://github.com/status-im/js-waku/issues/72).
|
||||
|
||||
By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets.
|
||||
We think JS-Waku builds a strong case for machine-to-machine (M2M) communications.
|
||||
The first use cases we are looking into are dApp notifications:
|
||||
Enabling dApp to notify their user directly in their wallets!
|
||||
Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified
|
||||
of events such as DAO voting.
|
||||
|
||||
We already have some POC in the pipeline to enable voting and polling on the Waku network,
|
||||
allowing users to save gas by **not** broadcasting each individual vote on the blockchain.
|
||||
|
||||
To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples
|
||||
of signing, validating, encrypting and decrypting messages using Web3.
|
||||
Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.
|
||||
|
||||
As you can read, we have grand plans for JS-Waku and Waku v2.
|
||||
There is a lot to do, and we would love some help so feel free to
|
||||
check out the new role in our team:
|
||||
[js-waku: Wallet & Dapp Integration Developer](https://status.im/our_team/jobs.html?gh_jid=3157894).
|
||||
We also have a number of [positions](https://status.im/our_team/jobs.html) open to work on Waku protocol and nim-waku.
|
||||
|
||||
If you are as excited as us by JS-Waku, why not build a dApp with it?
|
||||
You can find documentation on the [npmjs page](https://www.npmjs.com/package/js-waku).
|
||||
|
||||
Whether you are a developer, you can come chat with us using [WakuJS Web Chat](https://status-im.github.io/js-waku/)
|
||||
or [chat2](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md).
|
||||
You can get support on Discord [#waku-support (dev support)](https://discord.gg/VChNsDdj).
|
||||
If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2 in the browser</h2>
|
||||
<p class="post__text">
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web
|
||||
app to interact with the Waku v2 network. It is available right now on npm:
|
||||
</p>
|
||||
<div class="text-white post__text">
|
||||
<span class="bg-black px-2">const waku = await Waku.create({});</span>
|
||||
</div>
|
||||
<p class="post__text">
|
||||
As it is written in TypeScript, types are included in the npm package to
|
||||
allow easy integration with TypeScript, ClojureScript and other typed
|
||||
languages that compile to JavaScript.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Key Waku v2 protocols are already available:
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>message,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>store,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>relay</a
|
||||
>
|
||||
and
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>light push,</a
|
||||
>
|
||||
enabling your dApp to:
|
||||
</p>
|
||||
<ul class="post__text">
|
||||
<li>Send and receive near-instant messages on the Waku network (relay),</li>
|
||||
<li>
|
||||
Query nodes for messages that may have been missed, e.g. due to poor
|
||||
cellular network (store),
|
||||
</li>
|
||||
<li>Send messages with confirmations (light push).</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
|
|
@ -7,183 +7,100 @@ author: oskarth
|
|||
published: true
|
||||
permalink: /waku-v2-ethereum-messaging
|
||||
category: research
|
||||
summary: Talk from Taipei Ethereum Meetup. Read on to find out about our journey from Whisper to Waku v2, as well as how Waku v2 can be useful for Etherum Messaging.
|
||||
image: /assets/img/taipei_ethereum_meetup_slide.png
|
||||
discuss: https://forum.vac.dev/t/discussion-talk-vac-waku-v2-and-ethereum-messaging/60
|
||||
---
|
||||
|
||||
_The following post is a transcript of the talk given at the [Taipei Ethereum meetup, November 5](https://www.meetup.com/Taipei-Ethereum-Meetup/events/274033344/). There is also a [video recording](https://www.youtube.com/watch?v=lUDy1MoeYnI)._
|
||||
|
||||
---
|
||||
|
||||
## 0. Introduction
|
||||
|
||||
Hi! My name is Oskar and I'm the protocol research lead at Vac. This talk will be divided into two parts. First I'll talk about the journey from Whisper, to Waku v1 and now to Waku v2. Then I'll talk about messaging in Ethereum. After this talk, you should have an idea of what Waku v2 is, the problems it is trying to solve, as well as where it can be useful for messaging in Ethereum.
|
||||
|
||||
## PART 1 - VAC AND THE JOURNEY FROM WHISPER TO WAKU V1 TO WAKU V2
|
||||
|
||||
## 1. Vac intro
|
||||
|
||||
First, what is Vac? Vac grew out of our efforts Status to create a window on to Ethereum and secure messenger. Vac is modular protocol stack for p2p secure messaging, paying special attention to resource restricted devices, privacy and censorship resistance.
|
||||
|
||||
Today we are going to talk mainly about Waku v2, which is the transport privacy / routing aspect of the Vac protocol stack. It sits "above" the p2p overlay, such as libp2p dealing with transports etc, and below a conversational security layer dealing with messaging encryption, such as using Double Ratchet etc.
|
||||
|
||||
## 2. Whisper to Waku v1
|
||||
|
||||
In the beginning, there was Whisper. Whisper was part of the holy trinity of Ethereum. You had Ethereum for consensus/computation, Whisper for messaging, and Swarm for storage.
|
||||
|
||||
However, for various reasons, Whisper didn't get the attention it deserved. Development dwindled, it promised too much and it suffered from many issues, such as being extremely inefficient and not being suitable for running on e.g. mobile phone. Despite this, Status used it in its app from around 2017 to 2019. As far as I know, it was one of very few, if not the only, production uses of Whisper.
|
||||
|
||||
In an effort to solve some of its immediate problems, we forked Whisper into Waku and formalized it with a proper specification. This solved immediate bandwidth issues for light nodes, introduced rate limiting for better spam protection, improved historical message support, etc.
|
||||
|
||||
If you are interested in this journey, checkout the [EthCC talk Dean and I gave in Paris earlier this year](https://www.youtube.com/watch?v=6lLT33tsJjs).
|
||||
|
||||
Status upgraded to Waku v1 early 2020. What next?
|
||||
|
||||
## 3. Waku v1 to v2
|
||||
|
||||
We were far from done. The changes we had made were quite incremental and done in order to get tangible improvements as quickly as possible. This meant we couldn't address more fundamental issues related to full node routing scalability, running with libp2p for more transports, better security, better spam protection and incentivization.
|
||||
|
||||
This kickstarted Waku v2 efforts, which is what we've been working on since July. This work was and is initally centered around a few pieces:
|
||||
|
||||
(a) Moving to libp2p
|
||||
|
||||
(b) Better routing
|
||||
|
||||
(c) Accounting and user-run nodes
|
||||
|
||||
The general theme was: making the Waku network more scalable and robust.
|
||||
|
||||
We also did a scalability study to show at what point the network would run into issues, due to the inherent lack of routing that Whisper and Waku v1 provided.
|
||||
|
||||
You can read more about this [here](https://vac.dev/waku-v2-plan).
|
||||
|
||||
## 3.5 Waku v2 - Design goals
|
||||
|
||||
Taking a step back, what problem does Waku v2 attempt to solve compared to all the other solutions that exists out there? What type of applications should use it and why? We have the following design goals:
|
||||
|
||||
1. **Generalized messaging**. Many applications requires some form of messaging protocol to communicate between different subsystems or different nodes. This messaging can be human-to-human or machine-to-machine or a mix.
|
||||
|
||||
2. **Peer-to-peer**. These applications sometimes have requirements that make them suitable for peer-to-peer solutions.
|
||||
|
||||
3. **Resource restricted**. These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:
|
||||
|
||||
- limited bandwidth, CPU, memory, disk, battery, etc
|
||||
- not being publicly connectable
|
||||
- only being intermittently connected; mostly-offline
|
||||
|
||||
4. **Privacy**. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.
|
||||
|
||||
As well as to do so in a modular fashion. Meaning you can find a reasonable trade-off depending on your exact requirements. For example, you usually have to trade off some bandwidth to get metadata protection, and vice versa.
|
||||
|
||||
The concept of designing for resource restricted devices also leads to the concept of adaptive nodes, where you have more of a continuum between full nodes and light nodes. For example, if you switch your phone from mobile data to WiFi you might be able to handle more bandwidth, and so on.
|
||||
|
||||
## 4. Waku v2 - Breakdown
|
||||
|
||||
Where is Waku v2 at now, and how is it structured?
|
||||
|
||||
It is running over libp2p and we had our second internal testnet last week or so. As a side note, we name our testnets after subway stations in Taipei, the first one being Nangang, and the most recent one being Dingpu.
|
||||
|
||||
The main implementation is written in Nim using nim-libp2p, which is also powering Nimbus, an Ethereum 2 client. There is also a PoC for running Waku v2 in the browser. On a spec level, we have the following specifications that corresponds to the components that make up Waku v2:
|
||||
|
||||
- Waku v2 - this is the main spec that explains the goals of providing generalized messaging, in a p2p context, with a focus on privacy and running on resources restricted devices.
|
||||
- Relay - this is the main PubSub spec that provides better routing. It builds on top of GossipSub, which is what Eth2 heavily relies on as well.
|
||||
- Store - this is a 1-1 protocol for light nodes to get historical messages, if they are mostly-offline.
|
||||
- Filter - this is a 1-1 protocol for light nodes that are bandwidth restricted to only (or mostly) get messages they care about.
|
||||
- Message - this explains the payload, to get some basic encryption and content topics. It corresponds roughly to envelopes in Whisper/Waku v1.
|
||||
- Bridge - this explains how to do bridging between Waku v1 and Waku v2 for compatibility.
|
||||
|
||||
Right now, all protocols, with the exception of bridge, are in draft mode, meaning they have been implemented but are not yet being relied upon in production.
|
||||
|
||||
You can read more about the breakdown in this [update](https://vac.dev/waku-v2-update) though some progress has been made since then, as well was in the [main Waku v2 spec](https://specs.vac.dev/specs/waku/v2/waku-v2.html).
|
||||
|
||||
## 5. Waku v2 - Upcoming
|
||||
|
||||
What's coming up next? There are a few things.
|
||||
|
||||
For Status to use it in production, it needs to be integrated into the main app using the Nim Node API. The bridge also needs to be implemented and tested.
|
||||
|
||||
For other users, we are currently overhauling the API to allow usage from a browser, e.g. To make this experience great, there are also a few underlying infrastructure things that we need in nim-libp2p, such as a more secure HTTP server in Nim, Websockets and WebRTC support.
|
||||
|
||||
There are also some changes we made to at what level content encryption happens, and this needs to be made easier to use in the API. This means you can use a node without giving your keys to it, which is useful in some environments.
|
||||
|
||||
More generally, beyond getting to production-ready use, there are a few bigger pieces that we are working on or will work on soon. These are things like:
|
||||
|
||||
- Better scaling, by using topic sharding.
|
||||
- Accounting and user-run nodes, to account for and incentives full nodes.
|
||||
- Stronger and more rigorous privacy guarantees, e.g. through study of GossipSub, unlinkable packet formats, etc.
|
||||
- Rate Limit Nullifier for privacy preserving spam protection, a la what Barry Whitehat has presented before.
|
||||
|
||||
As well as better support for Ethereum M2M Messaging. Which is what I'll talk about next.
|
||||
|
||||
## PART 2 - ETHEREUM MESSAGING
|
||||
|
||||
A lot of what follows is inspired by exploratory work that John Lea has done at Status, previously Head of UX Architecture at Ubuntu.
|
||||
|
||||
## 6. Ethereum Messaging - Why?
|
||||
|
||||
It is easy to think that Waku v2 is only for human to human messaging, since that's how Waku is currently primarily used in the Status app. However, the goal is to be useful for generalized messaging, which includes other type of information as well as machine to machine messaging.
|
||||
|
||||
What is Ethereum M2M messaging? Going back to the Holy Trinity of Ethereum/Whisper/Swarm, the messaging component was seen as something that could facilitate messages between dapps and acts as a building block. This can help with things such as:
|
||||
|
||||
- Reducing on-chain transactions
|
||||
- Reduce latency for operations
|
||||
- Decentralize centrally coordinated services (like WalletConnect)
|
||||
- Improve UX of dapps
|
||||
- Broadcast live information
|
||||
- A message transport layer for state channels
|
||||
|
||||
And so on.
|
||||
|
||||
## 7. Ethereum Messaging - Why? (Cont)
|
||||
|
||||
What are some examples of practical things Waku as used for Ethereum Messaging could solve?
|
||||
|
||||
- Multisig transfers only needing one on chain transaction
|
||||
- DAO votes only needing one one chain transaction
|
||||
- Giving dapps ability to direct push notifications to users
|
||||
- Giving users ability to directly respond to requests from daps
|
||||
- Decentralized Wallet Connect
|
||||
|
||||
Etc.
|
||||
|
||||
## 8. What's needed to deliver this?
|
||||
|
||||
We can break it down into our actors:
|
||||
|
||||
- Decentralized M2M messaging system (Waku)
|
||||
- Native wallets (Argent, Metamask, Status, etc)
|
||||
- Dapps that benefit from M2M messaging
|
||||
- Users whose problems are being solved
|
||||
|
||||
Each of these has a bunch of requirements in turn. The messaging system needs to be decentralized, scalable, robust, etc. Wallets need support for messaging layer, dapps need to integrate this, etc.
|
||||
|
||||
This is a lot! Growing adoption is a challenge. There is a catch 22 in terms of justifying development efforts for wallets, when no dapps need it, and likewise for dapps when no wallets support Waku. In addition to this, there must be proven usage of Waku before it can be relied on, etc. How can we break this up into smaller pieces of work?
|
||||
|
||||
## 9. Breaking up the problem and a high level roadmap
|
||||
|
||||
We can start small. It doesn't and need to be used for critical features first. A more hybrid approach can be taken where it acts more as nice-to-haves.
|
||||
|
||||
1. Forking Whisper and solving scalablity, spam etc issues with it.
|
||||
This is a work in progress. What we talked about in part 1.
|
||||
2. Expose messaging API for Dapp developers.
|
||||
3. Implement decentralized version of WalletConnect.
|
||||
Currently wallets connect ot dapps with centralized service. Great UX.
|
||||
4. Solve DAO/Multi-Sig coordination problem.
|
||||
E.g. send message to wallet-derived key when it is time to sign a transaction.
|
||||
5. Extend dapp-to-user and user-to-dapp communication to more dapps.
|
||||
Use lessons learned and examples to drive adoptation for wallets/dapps.
|
||||
|
||||
And then build up from there.
|
||||
|
||||
## 10. We are hiring!
|
||||
|
||||
A lot of this will happen in Javascript and browsers, since that's the primarily environment for a lot of wallets and dapps. We are currently hiring for a Waku JS Wallet integration lead to help push this effort further.
|
||||
|
||||
Come talk to me after or [apply here](https://status.im/our_team/open_positions.html?gh_jid=2385338).
|
||||
|
||||
That's it! You can find us on Status, Telegram, vac.dev. I'm on twitter [here](https://twitter.com/oskarth).
|
||||
|
||||
Questions?
|
||||
|
||||
---
|
||||
<p class="post__intro">
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the
|
||||
browser. Our aim is to empower dApps with Waku v2, and it led to the creation
|
||||
of a new library. We believe now is good time to introduce it!
|
||||
</p>
|
||||
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2</h2>
|
||||
<p class="post__text">
|
||||
First, let’s review what Waku v2 is and what problem it is trying to solve.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution
|
||||
for the Status app to achieve decentralised communications on resource
|
||||
restricted devices (i.e., mobile phones). The Status chat feature was
|
||||
initially built over Whisper. However, Whisper has a number of caveats which
|
||||
makes it inefficient for mobile phones. For example, with Whisper, all
|
||||
devices are receiving all messages which is not ideal for limited data
|
||||
plans.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was
|
||||
introduced. To further enable web and restricted resource environments, Waku
|
||||
v2 was created based on libp2p. The migration of the Status chat feature to
|
||||
Waku v2 is currently in progress. We see the need of such solution in the
|
||||
broader Ethereum ecosystem, beyond Status.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
This is why we are building Waku v2 as a decentralised communication
|
||||
platform for all to use and build on. If you want to read more about Waku v2
|
||||
and what it aims to achieve, checkout What’s the Plan for Waku v2?.Since
|
||||
last year, we have been busy defining and implementing Waku v2 protocols in
|
||||
nim-waku, from which you can build wakunode2. Wakunode2 is an adaptive and
|
||||
modular Waku v2 node, it allows users to run their own node and use the Waku
|
||||
v2 protocols they need. The nim-waku project doubles as a library, that can
|
||||
be used to add Waku v2 support to native applications.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2 in the browser</h2>
|
||||
<p class="post__text">
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web
|
||||
app to interact with the Waku v2 network. It is available right now on npm:
|
||||
</p>
|
||||
<div class="text-white post__text">
|
||||
<span class="bg-black px-2">const waku = await Waku.create({});</span>
|
||||
</div>
|
||||
<p class="post__text">
|
||||
As it is written in TypeScript, types are included in the npm package to
|
||||
allow easy integration with TypeScript, ClojureScript and other typed
|
||||
languages that compile to JavaScript.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Key Waku v2 protocols are already available:
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>message,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>store,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>relay</a
|
||||
>
|
||||
and
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>light push,</a
|
||||
>
|
||||
enabling your dApp to:
|
||||
</p>
|
||||
<ul class="post__text">
|
||||
<li>Send and receive near-instant messages on the Waku network (relay),</li>
|
||||
<li>
|
||||
Query nodes for messages that may have been missed, e.g. due to poor
|
||||
cellular network (store),
|
||||
</li>
|
||||
<li>Send messages with confirmations (light push).</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
|
|
@ -7,226 +7,100 @@ author: sanaztaheri
|
|||
published: true
|
||||
permalink: /rln-relay
|
||||
category: research
|
||||
summary: This post is going to give you an overview of how spam protection can be achieved in Waku Relay through rate-limiting nullifiers. We will cover a summary of spam-protection methods in centralized and p2p systems, and the solution overview and details of the economic spam-protection method. The open issues and future steps are discussed in the end.
|
||||
image: /assets/img/rain.png
|
||||
discuss: https://forum.vac.dev/t/privacy-preserving-p2p-economic-spam-protection-in-waku-v2-with-rate-limiting-nullfiers/66
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
This post is going to give you an overview of how spam protection can be achieved in Waku Relay protocol[^2] through Rate-Limiting Nullifiers[^3] [^4] or RLN for short.
|
||||
|
||||
Let me give a little background about Waku(v2)[^1]. Waku is a privacy-preserving peer-to-peer (p2p) messaging protocol for resource-restricted devices. Being p2p means that Waku relies on **No** central server. Instead, peers collaboratively deliver messages in the network. Waku uses GossipSub[^16] as the underlying routing protocol (as of the writeup of this post). At a high level, GossipSub is based on publisher-subscriber architecture. That is, _peers, congregate around topics they are interested in and can send messages to topics. Each message gets delivered to all peers subscribed to the topic_. In GossipSub, a peer has a constant number of direct connections/neighbors. In order to publish a message, the author forwards its message to a subset of neighbors. The neighbors proceed similarly till the message gets propagated in the network of the subscribed peers. The message publishing and routing procedures are part of the Waku Relay[^17] protocol.
|
||||
|
||||
<p align="center">
|
||||
<img src="../assets/img/rln-relay/rln-relay-overview.png" width="200%" />
|
||||
<br />
|
||||
Figure 1: An overview of privacy-preserving p2p economic spam protection in Waku v2 RLN-Relay protocol.
|
||||
<p class="post__intro">
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the
|
||||
browser. Our aim is to empower dApps with Waku v2, and it led to the creation
|
||||
of a new library. We believe now is good time to introduce it!
|
||||
</p>
|
||||
|
||||
## What do we mean by spamming?
|
||||
|
||||
In centralized messaging systems, a spammer usually indicates an entity that uses the messaging system to send an unsolicited message (spam) to large numbers of recipients. However, in Waku with a p2p architecture, spam messages not only affect the recipients but also all the other peers involved in the routing process as they have to spend their computational power/bandwidth/storage capacity on processing spam messages. As such, we define a spammer as an entity that uses the messaging system to publish a large number of messages in a short amount of time. The messages issued in this way are called spam. In this definition, we disregard the intention of the spammer as well as the content of the message and the number of recipients.
|
||||
|
||||
## Possible Solutions
|
||||
|
||||
Has the spamming issue been addressed before? Of course yes! Here is an overview of the spam protection techniques with their trade-offs and use-cases. In this overview, we distinguish between protection techniques that are targeted for centralized messaging systems and those for p2p architectures.
|
||||
|
||||
### Centralized Messaging Systems
|
||||
|
||||
In traditional centralized messaging systems, spam usually signifies unsolicited messages sent in bulk or messages with malicious content like malware. Protection mechanisms include
|
||||
|
||||
- authentication through some piece of personally identifiable information e.g., phone number
|
||||
- checksum-based filtering to protect against messages sent in bulk
|
||||
- challenge-response systems
|
||||
- content filtering on the server or via a proxy application
|
||||
|
||||
These methods exploit the fact that the messaging system is centralized and a global view of the users' activities is available based on which spamming patterns can be extracted and defeated accordingly. Moreover, users are associated with an identifier e.g., a username which enables the server to profile each user e.g., to detect suspicious behavior like spamming. Such profiling possibility is against the user's anonymity and privacy.
|
||||
|
||||
Among the techniques enumerated above, authentication through phone numbers is a some-what economic-incentive measure as providing multiple valid phone numbers will be expensive for the attacker. Notice that while using an expensive authentication method can reduce the number of accounts owned by a single spammer, cannot address the spam issue entirely. This is because the spammer can still send bulk messages through one single account. For this approach to be effective, a centralized mediator is essential. That is why such a solution would not fit the p2p environments where no centralized control exists.
|
||||
|
||||
### P2P Systems
|
||||
|
||||
What about spam prevention in p2p messaging platforms? There are two techniques, namely _Proof of Work_[^8] deployed by Whisper[^9] and _Peer scoring_[^6] method (namely reputation-based approach) adopted by LibP2P. However, each of these solutions has its own shortcomings for real-life use-cases as explained below.
|
||||
|
||||
#### Proof of work
|
||||
|
||||
The idea behind the Proof Of Work i.e., POW[^8] is to make messaging a computationally costly operation hence lowering the messaging rate of **all** the peers including the spammers. In specific, the message publisher has to solve a puzzle and the puzzle is to find a nonce such that the hash of the message concatenated with the nonce has at least z leading zeros. z is known as the difficulty of the puzzle. Since the hash function is one-way, peers have to brute-force to find a nonce. Hashing is a computationally-heavy operation so is the brute-force. While solving the puzzle is computationally expensive, it is comparatively cheap to verify the solution.
|
||||
|
||||
POW is also used as the underlying mining algorithm in Ethereum and Bitcoin blockchain. There, the goal is to contain the mining speed and allow the decentralized network to come to a consensus, or agree on things like account balances and the order of transactions.
|
||||
|
||||
While the use of POW makes perfect sense in Ethereum / Bitcoin blockchain, it shows practical issues in heterogeneous p2p messaging systems with resource-restricted peers. Some peers won't be able to carry the designated computation and will be effectively excluded. Such exclusion showed to be practically an issue in applications like Status, which used to rely on POW for spam-protection, to the extent that the difficulty level had to be set close to zero.
|
||||
|
||||
#### Peer Scoring
|
||||
|
||||
The peer scoring method[^6] that is utilized by libp2p is to limit the number of messages issued by a peer in connection to another peer. That is each peer monitors all the peers to which it is directly connected and adjusts their messaging quota i.e., to route or not route their messages depending on their past activities. For example, if a peer detects its neighbor is sending more than x messages per month, can drop its quota to z.x where z is less than one. The shortcoming of this solution is that scoring is based on peers' local observations and the concept of the score is defined in relation to one single peer. This leaves room for an attack where a spammer can make connections to k peers in the system and publishes k.(x-1) messages by exploiting all of its k connections. Another attack scenario is through botnets consisting of a large number of e.g., a million bots. The attacker rents a botnet and inserts each of them as a legitimate peer to the network and each can publish x-1 messages per month[^7].
|
||||
|
||||
#### Economic-Incentive Spam protection
|
||||
|
||||
Is this the end of our spam-protection journey? Shall we simply give up and leave spammers be? Certainly not!
|
||||
Waku RLN-Relay gives us a p2p spam-protection method which:
|
||||
|
||||
- suits **p2p** systems and does not rely on any central entity.
|
||||
- is **efficient** i.e., with no unreasonable computational, storage, memory, and bandwidth requirement! as such, it fits the network of **heterogeneous** peers.
|
||||
- respects users **privacy** unlike reputation-based and centralized methods.
|
||||
- deploys **economic-incentives** to contain spammers' activity. Namely, there is a financial sacrifice for those who want to spam the system. How? follow along ...
|
||||
|
||||
We devise a general rule to save everyone's life and that is
|
||||
|
||||
**No one can publish more than M messages per epoch without being financially charged!**
|
||||
|
||||
We set M to 1 for now, but this can be any arbitrary value. You may be thinking "This is too restrictive! Only one per epoch?". Don't worry, we set the epoch to a reasonable value so that it does not slow down the communication of innocent users but will make the life of spammers harder! Epoch here can be every second, as defined by UTC date-time +-20s.
|
||||
|
||||
The remainder of this post is all about the story of how to enforce this limit on each user's messaging rate as well as how to impose the financial cost when the limit gets violated. This brings us to the Rate Limiting Nullifiers and how we integrate this technique into Waku v2 (in specific the Waku Relay protocol) to protect our valuable users against spammers.
|
||||
|
||||
# Technical Terms
|
||||
|
||||
**Zero-knowledge proof**: Zero-knowledge proof (ZKP)[^14] allows a _prover_ to show a _verifier_ that they know something, without revealing what that something is. This means you can do the trust-minimized computation that is also privacy-preserving. As a basic example, instead of showing your ID when going to a bar you simply give them proof that you are over 18, without showing the doorman your id. In this write-up, by ZKP we essentially mean zkSNARK[^15] which is one of the many types of ZKPs.
|
||||
|
||||
**Threshold Secret Sharing Scheme**: (m,n) Threshold secret-sharing is a method by which you can split a secret value s into n pieces in a way that the secret s can be reconstructed by having m pieces (m <= n). The economic-incentive spam protection utilizes a (2,n) secret sharing realized by Shamir Secret Sharing Scheme[^13].
|
||||
|
||||
# Overview: Economic-Incentive Spam protection through Rate Limiting Nullifiers
|
||||
|
||||
**Context**: We started the idea of economic-incentive spam protection more than a year ago and conducted a feasibility study to identify blockers and unknowns. The results are published in our prior [post](https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks). Since then major progress has been made and the prior identified blockers that are listed below are now addressed. Kudos to [Barry WhiteHat](https://github.com/barryWhiteHat), [Onur Kilic](https://github.com/kilic), [Koh Wei Jie](https://github.com/weijiekoh/perpetualpowersoftau) for all of their hard work, research, and development which made this progress possible.
|
||||
|
||||
- the proof time[^22] which was initially in the order of minutes ~10 mins and now is almost 0.5 seconds
|
||||
- the prover key size[^21] which was initially ~110MB and now is ~3.9MB
|
||||
- the lack of Shamir logic[^19] which is now implemented and part of the RLN repository[^4]
|
||||
- the concern regarding the potential multi-party computation for the trusted setup of zkSNARKs which got resolved[^20]
|
||||
- the lack of end-to-end integration that now we made it possible, have it implemented, and are going to present it in this post. New blockers are also sorted out during the e2e integration which we will discuss in the [Feasibility and Open Issues](#feasibility-and-open-issues) section.
|
||||
|
||||
Now that you have more context, let's see how the final solution works. The fundamental point is to make it economically costly to send more than your share of messages and to do so in a privacy-preserving and e2e fashion. To do that we have the following components:
|
||||
|
||||
- 1- **Group**: We manage all the peers inside a large group (later we can split peers into smaller groups, but for now consider only one). The group management is done via a smart contract which is devised for this purpose and is deployed on the Ethereum blockchain.
|
||||
- 2- **Membership**: To be able to send messages and in specific for the published messages to get routed by all the peers, publishing peers have to register to the group. Membership involves setting up public and private key pairs (think of it as the username and password). The private key remains at the user side but the public key becomes a part of the group information on the contract (publicly available) and everyone has access to it. Public keys are not human-generated (like usernames) and instead they are random numbers, as such, they do not reveal any information about the owner (think of public keys as pseudonyms). Registration is mandatory for the users who want to publish a message, however, users who only want to listen to the messages are more than welcome and do not have to register in the group.
|
||||
- **Membership fee**: Membership is not for free! each peer has to lock a certain amount of funds during the registration (this means peers have to have an Ethereum account with sufficient balance for this sake). This fund is safely stored on the contract and remains intact unless the peer attempts to break the rules and publish more than one message per epoch.
|
||||
- **Zero-knowledge Proof of membership**: Do you want your message to get routed to its destination, fine, but you have to prove that you are a member of the group (sorry, no one can escape the registration phase!). Now, you may be thinking that should I attach my public key to my message to prove my membership? Absolutely Not! we said that our solution respects privacy! membership proofs are done in a zero-knowledge manner that is each message will carry cryptographic proof asserting that "the message is generated by one of the current members of the group", so your identity remains private and your anonymity is preserved!
|
||||
- **Slashing through secret sharing**: Till now it does not seem like we can catch spammers, right? yes, you are right! now comes the exciting part, detecting spammers and slashing them. The core idea behind the slashing is that each publishing peer (not routing peers!) has to integrate a secret share of its private key inside the message. The secret share is deterministically computed over the private key and the current epoch. The content of this share is harmless for the peer's privacy (it looks random) unless the peer attempts to publish more than one message in the same epoch hence disclosing more than one secret share of its private key. Indeed two distinct shares of the private key under the same epoch are enough to reconstruct the entire private key. Then what should you do with the recovered private key? hurry up! go to the contract and withdraw the private key and claim its fund and get rich!! Are you thinking what if spammers attach junk values instead of valid secret shares? Of course, that wouldn't be cool! so, there is a zero-knowledge proof for this sake as well where the publishing peer has to prove that the secret shares are generated correctly.
|
||||
|
||||
A high-level overview of the economic spam protection is shown in Figure 1.
|
||||
|
||||
# Flow
|
||||
|
||||
In this section, we describe the flow of the economic-incentive spam detection mechanism from the viewpoint of a single peer. An overview of this flow is provided in Figure 3.
|
||||
|
||||
## Setup and Registration
|
||||
|
||||
A peer willing to publish a message is required to register. Registration is moderated through a smart contract deployed on the Ethereum blockchain. The state of the contract contains the list of registered members' public keys. An overview of registration is illustrated in Figure 2.
|
||||
|
||||
For the registration, a peer creates a transaction that sends x amount of Ether to the contract. The peer who has the "private key" `sk` associated with that deposit would be able to withdraw x Ether by providing valid proof. Note that `sk` is initially only known by the owning peer however it may get exposed to other peers in case the owner attempts spamming the system i.e., sending more than one message per epoch.
|
||||
The following relation holds between the `sk` and `pk` i.e., `pk = H(sk)` where `H` denotes a hash function.
|
||||
|
||||
<p align="center">
|
||||
<img src="../assets/img/rln-relay/rln-relay.png" />
|
||||
<br />
|
||||
Figure 2: Registration.
|
||||
</p>
|
||||
|
||||
## Maintaining the membership Merkle Tree
|
||||
|
||||
The ZKP of membership that we mentioned before relies on the representation of the entire group as a [Merkle Tree](). The tree construction and maintenance is delegated to the peers (the initial idea was to keep the tree on the chain as part of the contract, however, the cost associated with member deletion and insertion was high and unreasonable, please see [Feasibility and Open Issues](#Feasibility-and-Open-Issues) for more details). As such, each peer needs to build the tree locally and sync itself with the contract updates (peer insertion and deletion) to mirror them on its tree.
|
||||
Two pieces of information of the tree are important as they enable peers to generate zero-knowledge proofs. One is the root of the tree and the other is the membership proof (or the authentication path). The tree root is public information whereas the membership proof is private data (or more precisely the index of the peer in the tree).
|
||||
|
||||
## Publishing
|
||||
|
||||
In order to publish at a given epoch, each message must carry a proof i.e., a zero-knowledge proof signifying that the publishing peer is a registered member, and has not exceeded the messaging rate at the given epoch.
|
||||
|
||||
Recall that the enforcement of the messaging rate was through associating a secret shared version of the peer's `sk` into the message together with a ZKP that the secret shares are constructed correctly. As for the secret sharing part, the peer generates the following data:
|
||||
|
||||
1. `shareX`
|
||||
2. `shareY`
|
||||
3. `nullifier`
|
||||
|
||||
The pair (`shareX`, `shareY`) is the secret shared version of `sk` that are generated using Shamir secret sharing scheme. Having two such pairs for an identical `nullifier` results in full disclosure of peer's `sk` and hence burning the associated deposit. Note that the `nullifier` is a deterministic value derived from `sk` and `epoch` therefore any two messages issued by the same peer (i.e., using the same `sk`) for the same `epoch` are guaranteed to have identical `nullifier`s.
|
||||
|
||||
Finally, the peer generates a zero-knowledge proof `zkProof` asserting the membership of the peer in the group and the correctness of the attached secret share (`shareX`, `shareY`) and the `nullifier`. In order to generate a valid proof, the peer needs to have two private inputs i.e., its `sk` and its authentication path. Other inputs are the tree root, epoch, and the content of the message.
|
||||
|
||||
**Privacy Hint:** Note that the authentication path of each peer depends on the recent list of members (hence changes when new peers register or leave). As such, it is recommended (and necessary for privacy/anonymity) that the publisher updates her authentication path based on the latest status of the group and attempts the proof using the updated version.
|
||||
|
||||
An overview of the publishing procedure is provided in Figure 3.
|
||||
|
||||
## Routing
|
||||
|
||||
Upon the receipt of a message, the routing peer needs to decide whether to route it or not. This decision relies on the following factors:
|
||||
|
||||
1. If the epoch value attached to the message has a non-reasonable gap with the routing peer's current epoch then the message must be dropped (this is to prevent a newly registered peer spamming the system by messaging for all the past epochs).
|
||||
2. The message MUST contain valid proof that gets verified by the routing peer.
|
||||
If the preceding checks are passed successfully, then the message is relayed. In case of an invalid proof, the message is dropped. If spamming is detected, the publishing peer gets slashed (see [Spam Detection and Slashing](#Spam-Detection-and-Slashing)).
|
||||
|
||||
An overview of the routing procedure is provided in Figure 3.
|
||||
|
||||
### Spam Detection and Slashing
|
||||
|
||||
In order to enable local spam detection and slashing, routing peers MUST record the `nullifier`, `shareX`, and `shareY` of any incoming message conditioned that it is not spam and has valid proof. To do so, the peer should follow the following steps.
|
||||
|
||||
1. The routing peer first verifies the `zkProof` and drops the message if not verified.
|
||||
2. Otherwise, it checks whether a message with an identical `nullifier` has already been relayed.
|
||||
- a) If such message exists and its `shareX` and `shareY` components are different from the incoming message, then slashing takes place (if the `shareX` and `shareY` fields of the previously relayed message is identical to the incoming message, then the message is a duplicate and shall be dropped).
|
||||
- b) If none found, then the message gets relayed.
|
||||
|
||||
An overview of the slashing procedure is provided in Figure 3.
|
||||
|
||||
<p align="center">
|
||||
<img src="../assets/img/rln-relay/rln-message-verification.png" />
|
||||
<br />
|
||||
Figure 3: Publishing, Routing and Slashing workflow.
|
||||
</p>
|
||||
|
||||
# Feasibility and Open Issues
|
||||
|
||||
We've come a long way since a year ago, blockers resolved, now we have implemented it end-to-end. We learned lot and could identify further issues and unknowns some of which are blocking getting to production. The summary of the identified issues are presented below.
|
||||
|
||||
## Storage overhead per peer
|
||||
|
||||
Currently, peers are supposed to maintain the entire tree locally and it imposes storage overhead which is linear in the size of the group (see this [issue](https://github.com/vacp2p/research/issues/57)[^11] for more details). One way to cope with this is to use the light-node and full-node paradigm in which only a subset of peers who are more resourceful retain the tree whereas the light nodes obtain the necessary information by interacting with the full nodes. Another way to approach this problem is through a more storage efficient method (as described in this research issue[^12]) where peers store a partial view of the tree instead of the entire tree. Keeping the partial view lowers the storage complexity to O(log(N)) where N is the size of the group. There are still unknown unknowns to this solution, as such, it must be studied further to become fully functional.
|
||||
|
||||
## Cost-effective way of member insertion and deletion
|
||||
|
||||
Currently, the cost associated with RLN-Relay membership is around 30 USD[^10]. We aim at finding a more cost-effective approach. Please feel free to share with us your solution ideas in this regard in this [issue](https://github.com/vacp2p/research/issues/56).
|
||||
|
||||
## Exceeding the messaging rate via multiple registrations
|
||||
|
||||
While the economic-incentive solution has an economic incentive to discourage spamming, we should note that there is still **expensive attack(s)**[^23] that a spammer can launch to break the messaging rate limit. That is, the attacker can pay for multiple legit registrations e.g., k, hence being able to publish k messages per epoch. We believe that the higher the membership fee is, the less probable would be such an attack, hence a stronger level of spam-protection can be achieved. Following this argument, the high fee associated with the membership (which we listed above as an open problem) can indeed be contributing to a better protection level.
|
||||
|
||||
# Conclusion and Future Steps
|
||||
|
||||
As discussed in this post, Waku RLN Relay can achieve a privacy-preserving economic spam protection through rate-limiting nullifiers. The idea is to financially discourage peers from publishing more than one message per epoch. In specific, exceeding the messaging rate results in a financial charge. Those who violate this rule are called spammers and their messages are spam. The identification of spammers does not rely on any central entity. Also, the financial punishment of spammers is cryptographically guaranteed.
|
||||
In this solution, privacy is guaranteed since: 1) Peers do not have to disclose any piece of personally identifiable information in any phase i.e., neither in the registration nor in the messaging phase 2) Peers can prove that they have not exceeded the messaging rate in a zero-knowledge manner and without leaving any trace to their membership accounts.
|
||||
Furthermore, all the computations are light hence this solution fits the heterogenous p2p messaging system. Note that the zero-knowledge proof parts are handled through zkSNARKs and the benchmarking result can be found in the RLN benchmark report[^5].
|
||||
|
||||
**Future steps**: We are still at the PoC level, and the development is in progress. As our future steps,
|
||||
|
||||
- we would like to evaluate the running time associated with the Merkle tree operations. Indeed, the need to locally store Merkle tree on each peer was one of the unknowns discovered during this PoC and yet the concrete benchmarking result in this regard is not available.
|
||||
- We would also like to pursue our storage-efficient Merkle Tree maintenance solution in order to lower the storage overhead of peers.
|
||||
- In line with the storage optimization, the full-node light-node structure is another path to follow.
|
||||
- Another possible improvement is to replace the membership contract with a distributed group management scheme e.g., through distributed hash tables. This is to address possible performance issues that the interaction with the Ethereum blockchain may cause. For example, the registration transactions are subject to delay as they have to be mined before being visible in the state of the membership contract. This means peers have to wait for some time before being able to publish any message.
|
||||
|
||||
# Acknowledgement
|
||||
|
||||
Thanks to Onur Kılıç for his explanation and pointers and for assisting with development and runtime issues. Also thanks to Barry Whitehat for his time and insightful comments. Special thanks to Oskar Thoren for his constructive comments and his guides during the development of this PoC and the writeup of this post.
|
||||
|
||||
# References
|
||||
|
||||
[^1]: Waku v2: [https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-v2.md](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-v2.md)
|
||||
[^2]: RLN-Relay specifications: [https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-rln-relay.md](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-rln-relay.md)
|
||||
[^3]: RLN documentation: [https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?both)
|
||||
[^4]: RLN repositories: [https://github.com/kilic/RLN](https://github.com/kilic/RLN) and [https://github.com/kilic/rlnapp](https://github.com/kilic/rlnapp)
|
||||
[^5]: RLN Benchmark: [https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks](https://hackmd.io/tMTLMYmTR5eynw2lwK9n1w?view#Benchmarks)
|
||||
[^6]: Peer Scoring: [https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#peer-scoring)
|
||||
[^7]: Peer scoring security issues: [https://github.com/vacp2p/research/issues/44](https://github.com/vacp2p/research/issues/44)
|
||||
[^8]: Proof of work: [http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf](http://www.infosecon.net/workshop/downloads/2004/pdf/clayton.pdf) and [https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf](https://link.springer.com/content/pdf/10.1007/3-540-48071-4_10.pdf)
|
||||
[^9]: Whisper: [https://eips.ethereum.org/EIPS/eip-627](Whisper: https://eips.ethereum.org/EIPS/eip-627)
|
||||
[^10]: Cost-effective way of member insertion and deletion: [https://github.com/vacp2p/research/issues/56](https://github.com/vacp2p/research/issues/56)
|
||||
[^11]: Storage overhead per peer: [https://github.com/vacp2p/research/issues/57](https://github.com/vacp2p/research/issues/57)
|
||||
[^12]: Storage-efficient Merkle Tree maintenance: [https://github.com/vacp2p/research/pull/54](https://github.com/vacp2p/research/pull/54)
|
||||
[^13]: Shamir Secret Sharing Scheme: [https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing](https://en.wikipedia.org/wiki/Shamir%27s_Secret_Sharing)
|
||||
[^14]: Zero Knowledge Proof: [https://dl.acm.org/doi/abs/10.1145/3335741.3335750](https://dl.acm.org/doi/abs/10.1145/3335741.3335750) and [https://en.wikipedia.org/wiki/Zero-knowledge_proof](https://en.wikipedia.org/wiki/Zero-knowledge_proof)
|
||||
[^15]: zkSNARKs: [https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11](https://link.springer.com/chapter/10.1007/978-3-662-49896-5_11) and [https://coinpare.io/whitepaper/zcash.pdf](https://coinpare.io/whitepaper/zcash.pdf)
|
||||
[^16]: GossipSub: [https://docs.libp2p.io/concepts/publish-subscribe/](https://docs.libp2p.io/concepts/publish-subscribe/)
|
||||
[^17]: Waku Relay: [https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-relay.md](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-relay.md)
|
||||
[^18]: Prior blockers of RLN-Relay: [https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks](https://vac.dev/feasibility-semaphore-rate-limiting-zksnarks)
|
||||
[^19]: The lack of Shamir secret sharing in zkSNARKs: [https://github.com/vacp2p/research/issues/10](https://github.com/vacp2p/research/issues/10)
|
||||
[^20]: The MPC required for zkSNARKs trusted setup: [https://github.com/vacp2p/research/issues/9](https://github.com/vacp2p/research/issues/9)
|
||||
[^21]: Prover key size: [https://github.com/vacp2p/research/issues/8](https://github.com/vacp2p/research/issues/8)
|
||||
[^22]: zkSNARKs proof time: [https://github.com/vacp2p/research/issues/7](https://github.com/vacp2p/research/issues/7)
|
||||
[^23]: Attack on the messaging rate: [https://github.com/vacp2p/specs/issues/251](https://github.com/vacp2p/specs/issues/251)
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2</h2>
|
||||
<p class="post__text">
|
||||
First, let’s review what Waku v2 is and what problem it is trying to solve.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution
|
||||
for the Status app to achieve decentralised communications on resource
|
||||
restricted devices (i.e., mobile phones). The Status chat feature was
|
||||
initially built over Whisper. However, Whisper has a number of caveats which
|
||||
makes it inefficient for mobile phones. For example, with Whisper, all
|
||||
devices are receiving all messages which is not ideal for limited data
|
||||
plans.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was
|
||||
introduced. To further enable web and restricted resource environments, Waku
|
||||
v2 was created based on libp2p. The migration of the Status chat feature to
|
||||
Waku v2 is currently in progress. We see the need of such solution in the
|
||||
broader Ethereum ecosystem, beyond Status.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
This is why we are building Waku v2 as a decentralised communication
|
||||
platform for all to use and build on. If you want to read more about Waku v2
|
||||
and what it aims to achieve, checkout What’s the Plan for Waku v2?.Since
|
||||
last year, we have been busy defining and implementing Waku v2 protocols in
|
||||
nim-waku, from which you can build wakunode2. Wakunode2 is an adaptive and
|
||||
modular Waku v2 node, it allows users to run their own node and use the Waku
|
||||
v2 protocols they need. The nim-waku project doubles as a library, that can
|
||||
be used to add Waku v2 support to native applications.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2 in the browser</h2>
|
||||
<p class="post__text">
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web
|
||||
app to interact with the Waku v2 network. It is available right now on npm:
|
||||
</p>
|
||||
<div class="text-white post__text">
|
||||
<span class="bg-black px-2">const waku = await Waku.create({});</span>
|
||||
</div>
|
||||
<p class="post__text">
|
||||
As it is written in TypeScript, types are included in the npm package to
|
||||
allow easy integration with TypeScript, ClojureScript and other typed
|
||||
languages that compile to JavaScript.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Key Waku v2 protocols are already available:
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>message,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>store,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>relay</a
|
||||
>
|
||||
and
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>light push,</a
|
||||
>
|
||||
enabling your dApp to:
|
||||
</p>
|
||||
<ul class="post__text">
|
||||
<li>Send and receive near-instant messages on the Waku network (relay),</li>
|
||||
<li>
|
||||
Query nodes for messages that may have been missed, e.g. due to poor
|
||||
cellular network (store),
|
||||
</li>
|
||||
<li>Send messages with confirmations (light push).</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
|
|
@ -7,172 +7,100 @@ author: franck
|
|||
published: true
|
||||
permalink: /status-ipsum
|
||||
category: problem
|
||||
summary: "Status ipsum lorem exequoe"
|
||||
image: /assets/img/js-waku-gist.png
|
||||
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/82
|
||||
---
|
||||
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the browser.
|
||||
Our aim is to empower dApps with Waku v2, and it led to the creation of a new library.
|
||||
We believe now is good time to introduce it!
|
||||
<p class="post__intro">
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the
|
||||
browser. Our aim is to empower dApps with Waku v2, and it led to the creation
|
||||
of a new library. We believe now is good time to introduce it!
|
||||
</p>
|
||||
|
||||
## Waku v2
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2</h2>
|
||||
<p class="post__text">
|
||||
First, let’s review what Waku v2 is and what problem it is trying to solve.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution
|
||||
for the Status app to achieve decentralised communications on resource
|
||||
restricted devices (i.e., mobile phones). The Status chat feature was
|
||||
initially built over Whisper. However, Whisper has a number of caveats which
|
||||
makes it inefficient for mobile phones. For example, with Whisper, all
|
||||
devices are receiving all messages which is not ideal for limited data
|
||||
plans.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was
|
||||
introduced. To further enable web and restricted resource environments, Waku
|
||||
v2 was created based on libp2p. The migration of the Status chat feature to
|
||||
Waku v2 is currently in progress. We see the need of such solution in the
|
||||
broader Ethereum ecosystem, beyond Status.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
This is why we are building Waku v2 as a decentralised communication
|
||||
platform for all to use and build on. If you want to read more about Waku v2
|
||||
and what it aims to achieve, checkout What’s the Plan for Waku v2?.Since
|
||||
last year, we have been busy defining and implementing Waku v2 protocols in
|
||||
nim-waku, from which you can build wakunode2. Wakunode2 is an adaptive and
|
||||
modular Waku v2 node, it allows users to run their own node and use the Waku
|
||||
v2 protocols they need. The nim-waku project doubles as a library, that can
|
||||
be used to add Waku v2 support to native applications.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
First, let's review what Waku v2 is and what problem it is trying to solve.
|
||||
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised
|
||||
communications on resource restricted devices (i.e., mobile phones).
|
||||
|
||||
The Status chat feature was initially built over Whisper.
|
||||
However, Whisper has a number of caveats which makes it inefficient for mobile phones.
|
||||
For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.
|
||||
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced.
|
||||
To further enable web and restricted resource environments, Waku v2 was created based on libp2p.
|
||||
The migration of the Status chat feature to Waku v2 is currently in progress.
|
||||
|
||||
We see the need of such solution in the broader Ethereum ecosystem, beyond Status.
|
||||
This is why we are building Waku v2 as a decentralised communication platform for all to use and build on.
|
||||
If you want to read more about Waku v2 and what it aims to achieve,
|
||||
checkout [What's the Plan for Waku v2?](/waku-v2-plan).
|
||||
|
||||
Since last year, we have been busy defining and implementing Waku v2 protocols in [nim-waku](https://github.com/status-im/nim-waku),
|
||||
from which you can build [wakunode2](https://github.com/status-im/nim-waku#wakunode).
|
||||
Wakunode2 is an adaptive and modular Waku v2 node,
|
||||
it allows users to run their own node and use the Waku v2 protocols they need.
|
||||
The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.
|
||||
|
||||
## Waku v2 in the browser
|
||||
|
||||
We believe that dApps and wallets can benefit from the Waku network in several ways.
|
||||
For some dApps, it makes sense to enable peer-to-peer communications.
|
||||
For others, machine-to-machine communications would be a great asset.
|
||||
For example, in the case of a DAO,
|
||||
Waku could be used for gas-less voting.
|
||||
Enabling the DAO to notify their users of a new vote,
|
||||
and users to vote without interacting with the blockchain and spending gas.
|
||||
|
||||
[Murmur](https://github.com/status-im/murmur) was the first attempt to bring Whisper to the browser,
|
||||
acting as a bridge between devp2p and libp2p.
|
||||
Once Waku v2 was started and there was a native implementation on top of libp2p,
|
||||
a [chat POC](https://github.com/vacp2p/waku-web-chat) was created to demonstrate the potential of Waku v2
|
||||
in web environment.
|
||||
It showed how using js-libp2p with few modifications enabled access to the Waku v2 network.
|
||||
There was still some unresolved challenges.
|
||||
For example, nim-waku only support TCP connections which are not supported by browser applications.
|
||||
Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,
|
||||
which in turn could connect to wakunode2 via TCP.
|
||||
|
||||
However, to enable dApp and Wallet developers to easily integrate Waku in their product,
|
||||
we need to give them a library that is easy to use and works out of the box:
|
||||
introducing [JS-Waku](https://github.com/status-im/js-waku).
|
||||
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network.
|
||||
It is available right now on [npm](https://www.npmjs.com/package/js-waku):
|
||||
|
||||
`npm install js-waku`.
|
||||
|
||||
As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.
|
||||
|
||||
Key Waku v2 protocols are already available:
|
||||
[message](https://rfc.vac.dev/spec/14/), [store](https://rfc.vac.dev/spec/13/), [relay](https://rfc.vac.dev/spec/11/) and [light push](https://rfc.vac.dev/spec/19/),
|
||||
enabling your dApp to:
|
||||
|
||||
- Send and receive near-instant messages on the Waku network (relay),
|
||||
- Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
|
||||
- Send messages with confirmations (light push).
|
||||
|
||||
JS-Waku needs to operate in the same context from which Waku v2 was born:
|
||||
a restricted environment were connectivity or uptime are not guaranteed;
|
||||
JS-Waku brings Waku v2 to the browser.
|
||||
|
||||
## Achievements so far
|
||||
|
||||
We focused the past month on developing a [ReactJS Chat App](https://status-im.github.io/js-waku/).
|
||||
The aim was to create enough building blocks in JS-Waku to enable this showcase web app that
|
||||
we now [use for dogfooding](https://github.com/status-im/nim-waku/issues/399) purposes.
|
||||
|
||||
Most of the effort was on getting familiar with the [js-libp2p](https://github.com/libp2p/js-libp2p) library
|
||||
that we heavily rely on.
|
||||
JS-Waku is the second implementation of Waku v2 protocol,
|
||||
so a lot of effort on interoperability was needed.
|
||||
For example, to ensure compatibility with the nim-waku reference implementation,
|
||||
we run our [tests against wakunode2](https://github.com/status-im/js-waku/blob/90c90dea11dfd1277f530cf5d683fb92992fe141/src/lib/waku_relay/index.spec.ts#L137) as part of the CI.
|
||||
|
||||
This interoperability effort helped solidify the current Waku v2 specifications:
|
||||
By clarifying the usage of topics
|
||||
([#327](https://github.com/vacp2p/rfc/issues/327), [#383](https://github.com/vacp2p/rfc/pull/383)),
|
||||
fix discrepancies between specs and nim-waku
|
||||
([#418](https://github.com/status-im/nim-waku/issues/418), [#419](https://github.com/status-im/nim-waku/issues/419))
|
||||
and fix small nim-waku & nim-libp2p bugs
|
||||
([#411](https://github.com/status-im/nim-waku/issues/411), [#439](https://github.com/status-im/nim-waku/issues/439)).
|
||||
|
||||
To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.
|
||||
A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.
|
||||
Unfortunately websocket support is not yet available in [nim-libp2p](https://github.com/status-im/nim-libp2p/issues/407) so
|
||||
we ended up deploying [websockify](https://github.com/novnc/websockify) alongside wakunode2 instances.
|
||||
|
||||
As we built the [web chat app](https://github.com/status-im/js-waku/tree/main/examples/web-chat),
|
||||
we were able to fine tune the API to provide a simple and succinct interface.
|
||||
You can start a node, connect to other nodes and send a message in less than ten lines of code:
|
||||
|
||||
```javascript
|
||||
import { Waku } from "js-waku";
|
||||
|
||||
const waku = await Waku.create({});
|
||||
|
||||
const nodes = await getStatusFleetNodes();
|
||||
await Promise.all(nodes.map((addr) => waku.dial(addr)));
|
||||
|
||||
const msg = WakuMessage.fromUtf8String(
|
||||
"Here is a message!",
|
||||
"/my-cool-app/1/my-use-case/proto"
|
||||
);
|
||||
await waku.relay.send(msg);
|
||||
```
|
||||
|
||||
We have also put a bounty at [0xHack](https://0xhack.dev/) for using JS-Waku
|
||||
and running a [workshop](https://vimeo.com/551509621).
|
||||
We were thrilled to have a couple of hackers create new software using our libraries.
|
||||
One of the projects aimed to create a decentralised, end-to-end encrypted messenger app,
|
||||
similar to what the [ETH-DM](https://rfc.vac.dev/spec/20/) protocol aims to achieve.
|
||||
Another project was a decentralised Twitter platform.
|
||||
Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.
|
||||
|
||||
As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network
|
||||
while preserving this out of the box experience.
|
||||
|
||||
## What's next?
|
||||
|
||||
Next, we are directing our attention towards [Developer Experience](https://github.com/status-im/js-waku/issues/68).
|
||||
We already have [documentation](https://www.npmjs.com/package/js-waku) available but we want to provide more:
|
||||
[Tutorials](https://github.com/status-im/js-waku/issues/56), various examples
|
||||
and showing how [JS-Waku can be used with Web3](https://github.com/status-im/js-waku/issues/72).
|
||||
|
||||
By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets.
|
||||
We think JS-Waku builds a strong case for machine-to-machine (M2M) communications.
|
||||
The first use cases we are looking into are dApp notifications:
|
||||
Enabling dApp to notify their user directly in their wallets!
|
||||
Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified
|
||||
of events such as DAO voting.
|
||||
|
||||
We already have some POC in the pipeline to enable voting and polling on the Waku network,
|
||||
allowing users to save gas by **not** broadcasting each individual vote on the blockchain.
|
||||
|
||||
To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples
|
||||
of signing, validating, encrypting and decrypting messages using Web3.
|
||||
Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.
|
||||
|
||||
As you can read, we have grand plans for JS-Waku and Waku v2.
|
||||
There is a lot to do, and we would love some help so feel free to
|
||||
check out the new role in our team:
|
||||
[js-waku: Wallet & Dapp Integration Developer](https://status.im/our_team/jobs.html?gh_jid=3157894).
|
||||
We also have a number of [positions](https://status.im/our_team/jobs.html) open to work on Waku protocol and nim-waku.
|
||||
|
||||
If you are as excited as us by JS-Waku, why not build a dApp with it?
|
||||
You can find documentation on the [npmjs page](https://www.npmjs.com/package/js-waku).
|
||||
|
||||
Whether you are a developer, you can come chat with us using [WakuJS Web Chat](https://status-im.github.io/js-waku/)
|
||||
or [chat2](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md).
|
||||
You can get support on Discord [#waku-support (dev support)](https://discord.gg/VChNsDdj).
|
||||
If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2 in the browser</h2>
|
||||
<p class="post__text">
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web
|
||||
app to interact with the Waku v2 network. It is available right now on npm:
|
||||
</p>
|
||||
<div class="text-white post__text">
|
||||
<span class="bg-black px-2">const waku = await Waku.create({});</span>
|
||||
</div>
|
||||
<p class="post__text">
|
||||
As it is written in TypeScript, types are included in the npm package to
|
||||
allow easy integration with TypeScript, ClojureScript and other typed
|
||||
languages that compile to JavaScript.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Key Waku v2 protocols are already available:
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>message,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>store,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>relay</a
|
||||
>
|
||||
and
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>light push,</a
|
||||
>
|
||||
enabling your dApp to:
|
||||
</p>
|
||||
<ul class="post__text">
|
||||
<li>Send and receive near-instant messages on the Waku network (relay),</li>
|
||||
<li>
|
||||
Query nodes for messages that may have been missed, e.g. due to poor
|
||||
cellular network (store),
|
||||
</li>
|
||||
<li>Send messages with confirmations (light push).</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
|
|
@ -1,178 +0,0 @@
|
|||
---
|
||||
layout: post
|
||||
name: "Presenting JS-Waku: Waku v2 in the Browser"
|
||||
title: "Presenting JS-Waku: Waku v2 in the Browser"
|
||||
date: 2021-06-04 12:00:00 +0800
|
||||
author: franck
|
||||
published: true
|
||||
permalink: /presenting-js-waku
|
||||
category: research
|
||||
summary: "JS-Waku is bringing Waku v2 to the browser. Learn what we achieved so far and what is next in our pipeline!"
|
||||
image: /assets/img/js-waku-gist.png
|
||||
discuss: https://forum.vac.dev/t/discussion-presenting-js-waku-waku-v2-in-the-browser/81
|
||||
---
|
||||
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the browser.
|
||||
Our aim is to empower dApps with Waku v2, and it led to the creation of a new library.
|
||||
We believe now is good time to introduce it!
|
||||
|
||||
## Waku v2
|
||||
|
||||
First, let's review what Waku v2 is and what problem it is trying to solve.
|
||||
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution for the Status app to achieve decentralised
|
||||
communications on resource restricted devices (i.e., mobile phones).
|
||||
|
||||
The Status chat feature was initially built over Whisper.
|
||||
However, Whisper has a number of caveats which makes it inefficient for mobile phones.
|
||||
For example, with Whisper, all devices are receiving all messages which is not ideal for limited data plans.
|
||||
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was introduced.
|
||||
To further enable web and restricted resource environments, Waku v2 was created based on libp2p.
|
||||
The migration of the Status chat feature to Waku v2 is currently in progress.
|
||||
|
||||
We see the need of such solution in the broader Ethereum ecosystem, beyond Status.
|
||||
This is why we are building Waku v2 as a decentralised communication platform for all to use and build on.
|
||||
If you want to read more about Waku v2 and what it aims to achieve,
|
||||
checkout [What's the Plan for Waku v2?](/waku-v2-plan).
|
||||
|
||||
Since last year, we have been busy defining and implementing Waku v2 protocols in [nim-waku](https://github.com/status-im/nim-waku),
|
||||
from which you can build [wakunode2](https://github.com/status-im/nim-waku#wakunode).
|
||||
Wakunode2 is an adaptive and modular Waku v2 node,
|
||||
it allows users to run their own node and use the Waku v2 protocols they need.
|
||||
The nim-waku project doubles as a library, that can be used to add Waku v2 support to native applications.
|
||||
|
||||
## Waku v2 in the browser
|
||||
|
||||
We believe that dApps and wallets can benefit from the Waku network in several ways.
|
||||
For some dApps, it makes sense to enable peer-to-peer communications.
|
||||
For others, machine-to-machine communications would be a great asset.
|
||||
For example, in the case of a DAO,
|
||||
Waku could be used for gas-less voting.
|
||||
Enabling the DAO to notify their users of a new vote,
|
||||
and users to vote without interacting with the blockchain and spending gas.
|
||||
|
||||
[Murmur](https://github.com/status-im/murmur) was the first attempt to bring Whisper to the browser,
|
||||
acting as a bridge between devp2p and libp2p.
|
||||
Once Waku v2 was started and there was a native implementation on top of libp2p,
|
||||
a [chat POC](https://github.com/vacp2p/waku-web-chat) was created to demonstrate the potential of Waku v2
|
||||
in web environment.
|
||||
It showed how using js-libp2p with few modifications enabled access to the Waku v2 network.
|
||||
There was still some unresolved challenges.
|
||||
For example, nim-waku only support TCP connections which are not supported by browser applications.
|
||||
Hence, to connect to other node, the POC was connecting to a NodeJS proxy application using websockets,
|
||||
which in turn could connect to wakunode2 via TCP.
|
||||
|
||||
However, to enable dApp and Wallet developers to easily integrate Waku in their product,
|
||||
we need to give them a library that is easy to use and works out of the box:
|
||||
introducing [JS-Waku](https://github.com/status-im/js-waku).
|
||||
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web app to interact with the Waku v2 network.
|
||||
It is available right now on [npm](https://www.npmjs.com/package/js-waku):
|
||||
|
||||
`npm install js-waku`.
|
||||
|
||||
As it is written in TypeScript, types are included in the npm package to allow easy integration with TypeScript, ClojureScript and other typed languages that compile to JavaScript.
|
||||
|
||||
Key Waku v2 protocols are already available:
|
||||
[message](https://rfc.vac.dev/spec/14/), [store](https://rfc.vac.dev/spec/13/), [relay](https://rfc.vac.dev/spec/11/) and [light push](https://rfc.vac.dev/spec/19/),
|
||||
enabling your dApp to:
|
||||
|
||||
- Send and receive near-instant messages on the Waku network (relay),
|
||||
- Query nodes for messages that may have been missed, e.g. due to poor cellular network (store),
|
||||
- Send messages with confirmations (light push).
|
||||
|
||||
JS-Waku needs to operate in the same context from which Waku v2 was born:
|
||||
a restricted environment were connectivity or uptime are not guaranteed;
|
||||
JS-Waku brings Waku v2 to the browser.
|
||||
|
||||
## Achievements so far
|
||||
|
||||
We focused the past month on developing a [ReactJS Chat App](https://status-im.github.io/js-waku/).
|
||||
The aim was to create enough building blocks in JS-Waku to enable this showcase web app that
|
||||
we now [use for dogfooding](https://github.com/status-im/nim-waku/issues/399) purposes.
|
||||
|
||||
Most of the effort was on getting familiar with the [js-libp2p](https://github.com/libp2p/js-libp2p) library
|
||||
that we heavily rely on.
|
||||
JS-Waku is the second implementation of Waku v2 protocol,
|
||||
so a lot of effort on interoperability was needed.
|
||||
For example, to ensure compatibility with the nim-waku reference implementation,
|
||||
we run our [tests against wakunode2](https://github.com/status-im/js-waku/blob/90c90dea11dfd1277f530cf5d683fb92992fe141/src/lib/waku_relay/index.spec.ts#L137) as part of the CI.
|
||||
|
||||
This interoperability effort helped solidify the current Waku v2 specifications:
|
||||
By clarifying the usage of topics
|
||||
([#327](https://github.com/vacp2p/rfc/issues/327), [#383](https://github.com/vacp2p/rfc/pull/383)),
|
||||
fix discrepancies between specs and nim-waku
|
||||
([#418](https://github.com/status-im/nim-waku/issues/418), [#419](https://github.com/status-im/nim-waku/issues/419))
|
||||
and fix small nim-waku & nim-libp2p bugs
|
||||
([#411](https://github.com/status-im/nim-waku/issues/411), [#439](https://github.com/status-im/nim-waku/issues/439)).
|
||||
|
||||
To fully access the waku network, JS-Waku needs to enable web apps to connect to nim-waku nodes.
|
||||
A standard way to do so is using secure websockets as it is not possible to connect directly to a TCP port from the browser.
|
||||
Unfortunately websocket support is not yet available in [nim-libp2p](https://github.com/status-im/nim-libp2p/issues/407) so
|
||||
we ended up deploying [websockify](https://github.com/novnc/websockify) alongside wakunode2 instances.
|
||||
|
||||
As we built the [web chat app](https://github.com/status-im/js-waku/tree/main/examples/web-chat),
|
||||
we were able to fine tune the API to provide a simple and succinct interface.
|
||||
You can start a node, connect to other nodes and send a message in less than ten lines of code:
|
||||
|
||||
```javascript
|
||||
import { Waku } from "js-waku";
|
||||
|
||||
const waku = await Waku.create({});
|
||||
|
||||
const nodes = await getStatusFleetNodes();
|
||||
await Promise.all(nodes.map((addr) => waku.dial(addr)));
|
||||
|
||||
const msg = WakuMessage.fromUtf8String(
|
||||
"Here is a message!",
|
||||
"/my-cool-app/1/my-use-case/proto"
|
||||
);
|
||||
await waku.relay.send(msg);
|
||||
```
|
||||
|
||||
We have also put a bounty at [0xHack](https://0xhack.dev/) for using JS-Waku
|
||||
and running a [workshop](https://vimeo.com/551509621).
|
||||
We were thrilled to have a couple of hackers create new software using our libraries.
|
||||
One of the projects aimed to create a decentralised, end-to-end encrypted messenger app,
|
||||
similar to what the [ETH-DM](https://rfc.vac.dev/spec/20/) protocol aims to achieve.
|
||||
Another project was a decentralised Twitter platform.
|
||||
Such projects allow us to prioritize the work on JS-Waku and understand how DevEx can be improved.
|
||||
|
||||
As more developers use JS-Waku, we will evolve the API to allow for more custom and fine-tune usage of the network
|
||||
while preserving this out of the box experience.
|
||||
|
||||
## What's next?
|
||||
|
||||
Next, we are directing our attention towards [Developer Experience](https://github.com/status-im/js-waku/issues/68).
|
||||
We already have [documentation](https://www.npmjs.com/package/js-waku) available but we want to provide more:
|
||||
[Tutorials](https://github.com/status-im/js-waku/issues/56), various examples
|
||||
and showing how [JS-Waku can be used with Web3](https://github.com/status-im/js-waku/issues/72).
|
||||
|
||||
By prioritizing DevEx we aim to enable JS-Waku integration in dApps and wallets.
|
||||
We think JS-Waku builds a strong case for machine-to-machine (M2M) communications.
|
||||
The first use cases we are looking into are dApp notifications:
|
||||
Enabling dApp to notify their user directly in their wallets!
|
||||
Leveraging Waku as a decentralised infrastructure and standard so that users do not have to open their dApp to be notified
|
||||
of events such as DAO voting.
|
||||
|
||||
We already have some POC in the pipeline to enable voting and polling on the Waku network,
|
||||
allowing users to save gas by **not** broadcasting each individual vote on the blockchain.
|
||||
|
||||
To facilitate said applications, we are looking at improving integration with Web3 providers by providing examples
|
||||
of signing, validating, encrypting and decrypting messages using Web3.
|
||||
Waku is privacy conscious, so we will also provide signature and encryption examples decoupled from users' Ethereum identity.
|
||||
|
||||
As you can read, we have grand plans for JS-Waku and Waku v2.
|
||||
There is a lot to do, and we would love some help so feel free to
|
||||
check out the new role in our team:
|
||||
[js-waku: Wallet & Dapp Integration Developer](https://status.im/our_team/jobs.html?gh_jid=3157894).
|
||||
We also have a number of [positions](https://status.im/our_team/jobs.html) open to work on Waku protocol and nim-waku.
|
||||
|
||||
If you are as excited as us by JS-Waku, why not build a dApp with it?
|
||||
You can find documentation on the [npmjs page](https://www.npmjs.com/package/js-waku).
|
||||
|
||||
Whether you are a developer, you can come chat with us using [WakuJS Web Chat](https://status-im.github.io/js-waku/)
|
||||
or [chat2](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md).
|
||||
You can get support on Discord [#waku-support (dev support)](https://discord.gg/VChNsDdj).
|
||||
If you have any ideas on how Waku could enable a specific dapp or use case, do share, we are always keen to hear it.
|
|
@ -0,0 +1,106 @@
|
|||
---
|
||||
layout: post
|
||||
name: "Presenting JS-Waku: Waku v2 in the Browser"
|
||||
title: "Presenting JS-Waku: Waku v2 in the Browser"
|
||||
date: 2021-06-04 12:00:00 +0800
|
||||
author: franck
|
||||
published: true
|
||||
permalink: /presenting-js-waku
|
||||
category: research
|
||||
---
|
||||
|
||||
<p class="post__intro">
|
||||
For the past 3 months, we have been working on bringing Waku v2 to the
|
||||
browser. Our aim is to empower dApps with Waku v2, and it led to the creation
|
||||
of a new library. We believe now is good time to introduce it!
|
||||
</p>
|
||||
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2</h2>
|
||||
<p class="post__text">
|
||||
First, let’s review what Waku v2 is and what problem it is trying to solve.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Waku v2 comes from a need to have a more scalable, better optimised solution
|
||||
for the Status app to achieve decentralised communications on resource
|
||||
restricted devices (i.e., mobile phones). The Status chat feature was
|
||||
initially built over Whisper. However, Whisper has a number of caveats which
|
||||
makes it inefficient for mobile phones. For example, with Whisper, all
|
||||
devices are receiving all messages which is not ideal for limited data
|
||||
plans.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
To remediate this, a Waku mode (then Waku v1), based on devp2p, was
|
||||
introduced. To further enable web and restricted resource environments, Waku
|
||||
v2 was created based on libp2p. The migration of the Status chat feature to
|
||||
Waku v2 is currently in progress. We see the need of such solution in the
|
||||
broader Ethereum ecosystem, beyond Status.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
This is why we are building Waku v2 as a decentralised communication
|
||||
platform for all to use and build on. If you want to read more about Waku v2
|
||||
and what it aims to achieve, checkout What’s the Plan for Waku v2?.Since
|
||||
last year, we have been busy defining and implementing Waku v2 protocols in
|
||||
nim-waku, from which you can build wakunode2. Wakunode2 is an adaptive and
|
||||
modular Waku v2 node, it allows users to run their own node and use the Waku
|
||||
v2 protocols they need. The nim-waku project doubles as a library, that can
|
||||
be used to add Waku v2 support to native applications.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="post__block">
|
||||
<h2 class="post__subheading">Waku v2 in the browser</h2>
|
||||
<p class="post__text">
|
||||
JS-Waku is a JavaScript library that allows your dApp, wallet or other web
|
||||
app to interact with the Waku v2 network. It is available right now on npm:
|
||||
</p>
|
||||
<div class="text-white post__text">
|
||||
<span class="bg-black px-2">const waku = await Waku.create({});</span>
|
||||
</div>
|
||||
<p class="post__text">
|
||||
As it is written in TypeScript, types are included in the npm package to
|
||||
allow easy integration with TypeScript, ClojureScript and other typed
|
||||
languages that compile to JavaScript.
|
||||
</p>
|
||||
<p class="post__text">
|
||||
Key Waku v2 protocols are already available:
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>message,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>store,</a
|
||||
>
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>relay</a
|
||||
>
|
||||
and
|
||||
<a
|
||||
class="link link--body hover:opacity-50"
|
||||
href="https://vac.dev/"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>light push,</a
|
||||
>
|
||||
enabling your dApp to:
|
||||
</p>
|
||||
<ul class="post__text">
|
||||
<li>Send and receive near-instant messages on the Waku network (relay),</li>
|
||||
<li>
|
||||
Query nodes for messages that may have been missed, e.g. due to poor
|
||||
cellular network (store),
|
||||
</li>
|
||||
<li>Send messages with confirmations (light push).</li>
|
||||
</ul>
|
||||
</div>
|
Loading…
Reference in New Issue