vacp2p/vac.dev-experimental-old
1
0
Fork 0
mirror of https://github.com/vacp2p/vac.dev-experimental-old.git synced 2025-02-27 12:30:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
vac.dev-experimental-old/waku-v2-plan.html
Jenkins 8b5a45f199 Updates
2021-09-02 21:14:19 +00:00

796 lines
38 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html class="h-full" lang="en-US">
<head>
<title>Vac - What's the Plan for Waku v2?</title>
<meta charset="utf-8" />
<meta http-equiv="x-ua-compatible" content="ie=edge" />
<title>Vac</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link
rel="shortcut icon"
href="/assets/img/favicon.png"
type="image/png"
/>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link
href="https://fonts.googleapis.com/css2?family=Open+Sans:ital,wght@0,400;0,600;0,700;1,400;1,600&display=swap"
rel="stylesheet"
/>
<link rel="stylesheet" href="/assets/css/style.css" />
</head>
<body class="h-full flex flex-col">
<div
class="flex-grow container max-w-screen-xl mx-auto px-5 md:px-12 lg:pt-6"
>
<header>
<div class="container max-w-screen-xl sm:border-b">
<div
class="
nav-section
flex
justify-between
items-center
py-3
md:py-5
lg:py-10
"
>
<div class="logo md:pr-8 l:p-0">
<a href="/"
><img src="/assets/img/logo.png"
/></a>
</div>
<div class="flex justify-between items-center w-9/12">
<div class="burger block sm:hidden z-50">
<button
class="
burger__button burger__button--open
fixed
top-2
right-5
w-12
h-12
"
type="button"
aria-label="Mobile menu button"
>
<img
class="burger__icon"
src="/assets/img/burger.svg"
alt="Open menu button"
/>
</button>
<button
class="
burger__button burger__button--close
hidden
fixed
top-2
right-5
w-12
h-12
"
type="button"
aria-label="Close mobile menu button"
>
<img
class="burger__icon burger__icon--close"
src="/assets/img/close.svg"
alt="Close menu button"
/>
</button>
</div>
<nav class="nav max-w-screen-xm md:max-w-screen-sl container">
<ul
class="
nav__list
hidden
sm:flex
justify-between
container
text-xs
font-semibold
md:pr-8
l:p-0
"
>
<li class="hover:opacity-50">
<a class="nav__link" href="/#work">Work</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/#about">About</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/#join">Join VAC</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/research-log">Research log</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/media">Media</a>
</li>
<li class="hover:opacity-50">
<a href="https://specs.vac.dev" target="_blank" rel="noopener noreferrer"
>Specs</a
>
</li>
<li class="hover:opacity-50">
<a href="https://forum.vac.dev/" target="_blank" rel="noopener noreferrer"
>Forum</a
>
</li>
</ul>
</nav>
<ul class="social items-center hidden md:flex">
<li class="pr-5">
<a href="https://twitter.com/vacp2p" target="_blank" rel="noopener noreferrer"
><svg
width="25"
height="21"
viewBox="0 0 25 21"
fill="none"
xmlns="http://www.w3.org/2000/svg"
class="hover:opacity-50"
>
<path
d="M24.8872 3.04499C23.9872 3.43499 23.0572 3.70498 22.0672 3.82499C23.0872 3.22498 23.8672 2.26499 24.2272 1.09499C23.2672 1.66499 22.2172 2.05499 21.1072 2.29499C20.2072 1.33499 18.9172 0.734985 17.5072 0.734985C14.7772 0.734985 12.5872 2.95499 12.5872 5.65499C12.5872 6.04499 12.6172 6.40498 12.7072 6.76498C8.62721 6.58498 5.02721 4.60498 2.59721 1.63499C0.857207 4.75498 2.80721 7.33499 4.09721 8.20499C3.31721 8.20499 2.53721 7.96499 1.87721 7.60499C1.87721 10.035 3.58721 12.045 5.80721 12.495C5.32721 12.645 4.24721 12.735 3.58721 12.585C4.21721 14.535 6.04721 15.975 8.17721 16.005C6.49721 17.325 4.03721 18.375 0.887207 18.045C3.07721 19.455 5.65721 20.265 8.44721 20.265C17.5072 20.265 22.4272 12.765 22.4272 6.28499C22.4272 6.07499 22.4272 5.86499 22.3972 5.65499C23.4172 4.90499 24.2572 4.03499 24.8872 3.04499Z"
fill="#151512"
/>
</svg>
</a>
</li>
<li class="pr-5">
<a href="https://github.com/vacp2p" target="_blank" rel="noopener noreferrer"
><svg
width="26"
height="25"
viewBox="0 0 26 25"
fill="none"
xmlns="http://www.w3.org/2000/svg"
class="hover:opacity-50"
>
<path
d="M12.8857 0.856567C6.26021 0.856567 0.915339 6.20154 0.950043 12.7951C0.9778 18.0687 4.43935 22.5427 9.21766 24.1227C9.81824 24.2327 10.0353 23.864 10.0336 23.5474C10.0321 23.2635 10.0177 22.5129 10.0065 21.5171C6.67274 22.238 5.95552 19.9163 5.95552 19.9163C5.40376 18.5369 4.61433 18.1698 4.61433 18.1698C3.51994 17.4296 4.69151 17.4444 4.69151 17.4444C5.89646 17.5291 6.53549 18.6751 6.53549 18.6751C7.61609 20.4989 9.35182 19.9727 10.0342 19.6665C10.1382 18.8951 10.4459 18.3689 10.7878 18.0702C8.12222 17.7684 5.31483 16.7443 5.29076 12.1708C5.2839 10.8672 5.74629 9.80152 6.50989 8.96619C6.3838 8.66445 5.96641 7.45009 6.61027 5.80766C6.61027 5.80766 7.61658 5.4866 9.9167 7.03094C10.8723 6.76636 11.8976 6.63408 12.9191 6.62962C13.9376 6.63556 14.9658 6.76636 15.9257 7.03242C18.2081 5.48809 19.2163 5.80914 19.2163 5.80914C19.8789 7.45306 19.4743 8.66594 19.3529 8.96767C20.1268 9.80301 20.5959 10.8687 20.6028 12.1723C20.6269 16.7577 17.8272 17.767 15.1558 18.0628C15.5882 18.4314 15.976 19.1597 15.9819 20.273C15.9903 21.8693 15.9821 23.1565 15.9841 23.5474C15.9858 23.867 16.2038 24.2386 16.8122 24.1212C21.5663 22.5397 24.9778 18.0672 24.95 12.7951C24.9153 6.20154 19.5142 0.856567 12.8857 0.856567Z"
fill="#151512"
/>
</svg>
</a>
</li>
<li>
<a
href="https://discord.gg/PQFdubGt6d"
target="_blank"
rel="noopener noreferrer"
><svg
width="25"
height="21"
viewBox="0 0 25 21"
fill="none"
xmlns="http://www.w3.org/2000/svg"
class="hover:opacity-50"
>
<path
d="M22.7861 9.04256C21.8482 5.74455 20.7799 4.04048 20.7627 4.00991C20.7017 3.93459 19.189 2.104 15.5271 0.75L15.0353 2.0764C16.7774 2.72057 18.0116 3.50643 18.6899 4.01419C16.6599 3.40408 14.2431 3.03041 12.1008 3.03041C9.95851 3.03041 7.53775 3.40408 5.50128 4.01419C6.18496 3.50648 7.42744 2.72057 9.17631 2.0764L8.69846 0.75C5.02238 2.104 3.49044 3.93459 3.42863 4.00991C3.41108 4.04048 2.32479 5.74455 1.35221 9.04256C0.414855 12.2208 0.0415214 16.7045 0.027872 16.8843C0.109225 17.0131 1.97891 20.25 7.12077 20.25L8.43406 18.3536C6.97595 17.964 5.58693 17.3357 4.31689 16.4832L5.10228 15.3069C7.15122 16.6822 9.54509 17.4092 12.0251 17.4092C14.5051 17.4092 16.9067 16.6822 18.9701 15.3069L19.7431 16.4832C18.4641 17.3357 17.0684 17.964 15.6062 18.3536L16.8995 20.25C22.0414 20.25 23.9452 17.0131 24.0279 16.8843C24.0161 16.7045 23.69 12.2208 22.7861 9.04256ZM8.79853 12.7392H7.39228L7.40468 10.3841H8.81093L8.79853 12.7392ZM16.7071 12.7392H15.3008L15.3132 10.3841H16.7195L16.7071 12.7392Z"
fill="#151512"
/>
</svg>
</a>
</li>
</ul>
</div>
<div
class="
overlay
container
max-w-screen-sm
w-full
hidden
sm:hidden
fixed
top-0
right-0
h-screen
bg-black bg-opacity-40
z-30
"
>
<nav
class="
nav-mobile
hidden
fixed
top-0
right-0
flex flex-col
justify-between
items-center
pt-14
px-12
pb-5
bg-white
w-9/12
h-3/4
z-40
"
>
<ul
class="
nav__list
flex flex-col flex-1
justify-between
items-center
container
box-content
w-32
h-auto
max-h-nav
text-xs
font-normal
"
>
<li class="hover:opacity-50">
<a class="nav__link" href="/#work">Work</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/#about">About</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/#join">Join VAC</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/research-log">Research log</a>
</li>
<li class="hover:opacity-50">
<a class="nav__link" href="/media">Media</a>
</li>
<li class="hover:opacity-50">
<a href="https://specs.vac.dev" target="_blank" rel="noopener noreferrer"
>Specs</a
>
</li>
<li class="hover:opacity-50">
<a href="https://forum.vac.dev/" target="_blank" rel="noopener noreferrer"
>Forum</a
>
</li>
</ul>
<ul class="social items-center flex mt-8">
<li class="pr-5">
<a href="https://twitter.com/vacp2p" target="_blank" rel="noopener noreferrer"
><svg
width="25"
height="21"
viewBox="0 0 25 21"
fill="none"
xmlns="http://www.w3.org/2000/svg"
class="hover:opacity-50"
>
<path
d="M24.8872 3.04499C23.9872 3.43499 23.0572 3.70498 22.0672 3.82499C23.0872 3.22498 23.8672 2.26499 24.2272 1.09499C23.2672 1.66499 22.2172 2.05499 21.1072 2.29499C20.2072 1.33499 18.9172 0.734985 17.5072 0.734985C14.7772 0.734985 12.5872 2.95499 12.5872 5.65499C12.5872 6.04499 12.6172 6.40498 12.7072 6.76498C8.62721 6.58498 5.02721 4.60498 2.59721 1.63499C0.857207 4.75498 2.80721 7.33499 4.09721 8.20499C3.31721 8.20499 2.53721 7.96499 1.87721 7.60499C1.87721 10.035 3.58721 12.045 5.80721 12.495C5.32721 12.645 4.24721 12.735 3.58721 12.585C4.21721 14.535 6.04721 15.975 8.17721 16.005C6.49721 17.325 4.03721 18.375 0.887207 18.045C3.07721 19.455 5.65721 20.265 8.44721 20.265C17.5072 20.265 22.4272 12.765 22.4272 6.28499C22.4272 6.07499 22.4272 5.86499 22.3972 5.65499C23.4172 4.90499 24.2572 4.03499 24.8872 3.04499Z"
fill="#151512"
/>
</svg>
</a>
</li>
<li class="pr-5">
<a href="https://github.com/vacp2p" target="_blank" rel="noopener noreferrer"
><svg
width="26"
height="25"
viewBox="0 0 26 25"
fill="none"
xmlns="http://www.w3.org/2000/svg"
class="hover:opacity-50"
>
<path
d="M12.8857 0.856567C6.26021 0.856567 0.915339 6.20154 0.950043 12.7951C0.9778 18.0687 4.43935 22.5427 9.21766 24.1227C9.81824 24.2327 10.0353 23.864 10.0336 23.5474C10.0321 23.2635 10.0177 22.5129 10.0065 21.5171C6.67274 22.238 5.95552 19.9163 5.95552 19.9163C5.40376 18.5369 4.61433 18.1698 4.61433 18.1698C3.51994 17.4296 4.69151 17.4444 4.69151 17.4444C5.89646 17.5291 6.53549 18.6751 6.53549 18.6751C7.61609 20.4989 9.35182 19.9727 10.0342 19.6665C10.1382 18.8951 10.4459 18.3689 10.7878 18.0702C8.12222 17.7684 5.31483 16.7443 5.29076 12.1708C5.2839 10.8672 5.74629 9.80152 6.50989 8.96619C6.3838 8.66445 5.96641 7.45009 6.61027 5.80766C6.61027 5.80766 7.61658 5.4866 9.9167 7.03094C10.8723 6.76636 11.8976 6.63408 12.9191 6.62962C13.9376 6.63556 14.9658 6.76636 15.9257 7.03242C18.2081 5.48809 19.2163 5.80914 19.2163 5.80914C19.8789 7.45306 19.4743 8.66594 19.3529 8.96767C20.1268 9.80301 20.5959 10.8687 20.6028 12.1723C20.6269 16.7577 17.8272 17.767 15.1558 18.0628C15.5882 18.4314 15.976 19.1597 15.9819 20.273C15.9903 21.8693 15.9821 23.1565 15.9841 23.5474C15.9858 23.867 16.2038 24.2386 16.8122 24.1212C21.5663 22.5397 24.9778 18.0672 24.95 12.7951C24.9153 6.20154 19.5142 0.856567 12.8857 0.856567Z"
fill="#151512"
/>
</svg>
</a>
</li>
<li>
<a
href="https://discord.gg/PQFdubGt6d"
target="_blank"
rel="noopener noreferrer"
><svg
width="25"
height="21"
viewBox="0 0 25 21"
fill="none"
xmlns="http://www.w3.org/2000/svg"
class="hover:opacity-50"
>
<path
d="M22.7861 9.04256C21.8482 5.74455 20.7799 4.04048 20.7627 4.00991C20.7017 3.93459 19.189 2.104 15.5271 0.75L15.0353 2.0764C16.7774 2.72057 18.0116 3.50643 18.6899 4.01419C16.6599 3.40408 14.2431 3.03041 12.1008 3.03041C9.95851 3.03041 7.53775 3.40408 5.50128 4.01419C6.18496 3.50648 7.42744 2.72057 9.17631 2.0764L8.69846 0.75C5.02238 2.104 3.49044 3.93459 3.42863 4.00991C3.41108 4.04048 2.32479 5.74455 1.35221 9.04256C0.414855 12.2208 0.0415214 16.7045 0.027872 16.8843C0.109225 17.0131 1.97891 20.25 7.12077 20.25L8.43406 18.3536C6.97595 17.964 5.58693 17.3357 4.31689 16.4832L5.10228 15.3069C7.15122 16.6822 9.54509 17.4092 12.0251 17.4092C14.5051 17.4092 16.9067 16.6822 18.9701 15.3069L19.7431 16.4832C18.4641 17.3357 17.0684 17.964 15.6062 18.3536L16.8995 20.25C22.0414 20.25 23.9452 17.0131 24.0279 16.8843C24.0161 16.7045 23.69 12.2208 22.7861 9.04256ZM8.79853 12.7392H7.39228L7.40468 10.3841H8.81093L8.79853 12.7392ZM16.7071 12.7392H15.3008L15.3132 10.3841H16.7195L16.7071 12.7392Z"
fill="#151512"
/>
</svg>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
</header>
<main class="bg-white text-black flex flex-col"><section
class="
container
max-w-screen-xl
flex flex-col
sm:flex-row
pt-10
pb-0
md:pb-10
lg:pb-0
"
>
<div
class="
heading-block
w-full
sm:w-2/12
lg:w-3/12
flex
lg:justify-center
items-start
pb-3
sm:pb-0
"
>
<a class="link link--back" href="/research-log/">Back</a>
</div>
<div
class="info-block w-full sm:w-10/12 lg:w-9/12 pb-5 sm:pb-10 overflow-hidden"
>
<div class="post mb-10">
<h1 class="text-xl md:text-xxl mb-5 sm:max-w-md lg:max-w-2xl">
What's the Plan for Waku v2?
</h1>
<div>
<span class="text-s lg:text-base">
01 Jul 2020 • by
</span>
<a
href="/authors/oskarth"
class="text-s lg:text-base font-bold hover:underline"
>oskarth</a
>
</div>
</div>
<div class="post__content"><p><strong>tldr: The Waku network is fragile and doesn’t scale. Here’s how to solve it.</strong></p>
<p><em>NOTE: This post was originally written with Status as a primary use case in mind, which reflects how we talk about some problems here. However, Waku v2 is a general-purpose private p2p messaging protocol, especially for people running in resource restricted environments.</em></p>
<h1 id="problem">Problem</h1>
<p>The Waku network is fragile and doesn’t scale.</p>
<p>As <a href="https://status.im">Status</a> is moving into a user-acquisition phase and is improving retention rates for users they need the infrastructure to keep up, specifically when it comes to messaging.</p>
<p>Based on user acquisition models, the initial goal is to support 100k DAU in September, with demand growing from there.</p>
<p>With the Status Scaling Model we have studied the current bottlenecks as a function of concurrent users (CCU) and daily active users (DAU). Here are the conclusions.</p>
<p>*<strong>*1. Connection limits**</strong>. With 100 full nodes we reach ~10k CCU based on connection limits. This can primarily be addressed by increasing the number of nodes (cluster or user operated). This assumes node discovery works. It is also worth investigating the limitations of max number of connections, though this is likely to be less relevant for user-operated nodes. For a user-operated network, this means 1% of users have to run a full node. See Fig 1-2.</p>
<p>*<strong>*2. Bandwidth as a bottleneck**</strong>. We notice that memory usage appears to not be
the primary bottleneck for full nodes, and the bottleneck is still bandwidth. To support 10k DAU, and full nodes with an amplification factor of 25 the required Internet speed is ~50 Mbps, which is a fast home Internet connection. For ~100k DAU only cloud-operated nodes can keep up (500 Mbps). See Fig 3-5.</p>
<p>*<strong>*3. Amplification factors**</strong>. Reducing amplification factors with better routing, would have a high impact, but it is likely we’d need additional measures as well, such as topic sharding or similar. See Fig 8-13.</p>
<p>Figure 1-5:</p>
<p><img src="assets/img/status_scaling_model_fig1.png" alt="" />
<img src="assets/img/status_scaling_model_fig2.png" alt="" />
<img src="assets/img/status_scaling_model_fig3.png" alt="" />
<img src="assets/img/status_scaling_model_fig4.png" alt="" />
<img src="assets/img/status_scaling_model_fig5.png" alt="" /></p>
<p>See <a href="https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0">https://colab.research.google.com/drive/1Fz-oxRxxAFPpM1Cowpnb0nT52V1-yeRu#scrollTo=Yc3417FUJJ_0</a> for the full report.</p>
<p>What we need to do is:</p>
<ol>
<li>Reduce amplification factors</li>
<li>Get more user-run full nodes</li>
</ol>
<p>Doing this means the Waku network will be able to scale, and doing so in the right way, in a robust fashion. What would a fragile way of scaling be? Increasing our reliance on a Status Pte Ltd operated cluster which would paint us in a corner where we:</p>
<ul>
<li>keep increasing requirements for Internet speed for full nodes</li>
<li>are vulnerable to censorship and attacks</li>
<li>have to control the topology in an artifical manner to keep up with load</li>
<li>basically re-invent a traditional centralized client-server app with extra steps</li>
<li>deliberately ignore most of our principles</li>
<li>risk the network being shut down when we run out of cash</li>
</ul>
<h1 id="appetite">Appetite</h1>
<p>Our initial risk appetite for this is 6 weeks for a small team.</p>
<p>The idea is that we want to make tangible progress towards the goal in a limited period of time, as opposed to getting bogged down in trying to find a theoretically perfect generalized solution. Fixed time, variable scope.</p>
<p>It is likely some elements of a complete solution will be done separately. See later sections for that.</p>
<h1 id="solution">Solution</h1>
<p>There are two main parts of the solution. One is to reduce amplification factors, and the other is incentivization to get more user run full nodes with desktop, etc.</p>
<p>What does a full node provide? It provides connectivity to the network, can act as a bandwidth “barrier” and be high or reasonably high availability. What this means right now is essentially topic interest and storing historical messages.</p>
<p>The goal is here to improve the status quo, not get a perfect solution from the get go. All of this can be iterated on further, for stronger guarantees, as well as replaced by other new modules.</p>
<p>Let’s first look at the baseline, and then go into some of the tracks and their phases. Track 1 is best done first, after which track 2 and 3 can be executed in parallel. Track 1 gives us more options for track 2 and 3. The work in track 1 is currently more well-defined, so it is likely the specifics of track 2 and 3 will get refined at a later stage.</p>
<h2 id="baseline">Baseline</h2>
<p>Here’s where we are at now. In reality, the amplification factor are likely even worse than this (15 in the graph below), up to 20-30. Especially with an open network, where we can’t easily control connectivity and availability of nodes. Left unchecked, with a full mesh, it could even go as high x100, though this is likely excessive and can be dialed down. See scaling model for more details.</p>
<p><img src="assets/img/waku_v1_routing_small.png" alt="" /></p>
<h2 id="track-1---move-to-libp2p">Track 1 - Move to libp2p</h2>
<p>Moving to PubSub over libp2p wouldn’t improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.</p>
<p>See more on libp2p PubSub here: <a href="https://docs.libp2p.io/concepts/publish-subscribe/">https://docs.libp2p.io/concepts/publish-subscribe/</a></p>
<p>As part of this move, there are a few individual pieces that are needed.</p>
<h3 id="1-floodsub">1. FloodSub</h3>
<p>This is essentially what Waku over libp2p would look like in its most basic form.</p>
<p>One difference that is worth noting is that the app topics would <strong>not</strong> be the same as Waku topics. Why? In Waku we currently don’t use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.</p>
<p>Why can’t we use Waku topics for routing directly? PubSub over libp2p isn’t built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.</p>
<p><img src="assets/img/waku_v2_routing_flood_small.png" alt="" /></p>
<p>Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.</p>
<p>Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: <a href="https://github.com/status-im/nim-waku">https://github.com/status-im/nim-waku</a></p>
<h3 id="2-simplify-the-protocol">2. Simplify the protocol</h3>
<p>Due to Waku’s origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (<a href="https://specs.vac.dev/specs/waku/waku.html">https://specs.vac.dev/specs/waku/waku.html</a>). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?</p>
<p>Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.</p>
<p>The essential characteristic that has to be maintained is that we don’t need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.</p>
<h3 id="3-core-integration">3. Core integration</h3>
<p>As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.</p>
<h3 id="4-topic-interest-behavior">4. Topic interest behavior</h3>
<p>While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (<a href="https://vac.dev/fixing-whisper-with-waku">https://vac.dev/fixing-whisper-with-waku</a>). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.</p>
<h3 id="5-historical-message-caching">5. Historical message caching</h3>
<p>Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there’s no reason to reimplement a poor API if we don’t have to.</p>
<p>Also see section below on adaptive nodes and capabilities.</p>
<h3 id="6-waku-v1--libp2p-bridge">6. Waku v1 &lt;&gt; Libp2p bridge</h3>
<p>To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.</p>
<h2 id="track-2---better-routing">Track 2 - Better routing</h2>
<p>This is where we improve the amplification factors.</p>
<h3 id="1-gossipsub">1. GossipSub</h3>
<p>This is a subprotocol of FloodSub in the libp2p world. Moving to GossipSub would allow traffic between full nodes to go from an amplification factor of ~25 to ~6. This basically creates a mesh of stable bidirectional connections, together with some gossiping capabilities outside of this view.</p>
<p>Explaining how GossipSub works is out of scope of this document. It is implemented in nim-libp2p and used by Nimbus as part of Eth2. You can read the specs here in more detail if you are interested: <a href="https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md">https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md</a> and <a href="https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md">https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md</a></p>
<p><img src="assets/img/waku_v2_routing_gossip_small.png" alt="" /></p>
<p><img src="assets/img/status_scaling_model_fig8.png" alt="" />
<img src="assets/img/status_scaling_model_fig9.png" alt="" />
<img src="assets/img/status_scaling_model_fig10.png" alt="" />
<img src="assets/img/status_scaling_model_fig11.png" alt="" /></p>
<p>While we technically could implement this over existing Waku, we’d have to re-implement it, and we’d lose out on all the other benefits libp2p would provide, as well as the ecosystem of people and projects working on improving the scalability and security of these protocols.</p>
<h3 id="2-topic-sharding">2. Topic sharding</h3>
<p>This one is slightly more speculative in terms of its ultimate impact. The basic idea is to split the application topic into N shards, say 10, and then each full node can choose which shards to listen to. This can reduce amplification factors by another factor of 10.</p>
<p><img src="assets/img/waku_v2_routing_sharding_small.png" alt="" /></p>
<p><img src="assets/img/status_scaling_model_fig12.png" alt="" />
<img src="assets/img/status_scaling_model_fig13.png" alt="" /></p>
<p>Note that this means a light node that listens to several topics would have to be connected to more full nodes to get connectivity. For a more exotic version of this, see <a href="https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47">https://forum.vac.dev/t/rfc-topic-propagation-extension-to-libp2p-pubsub/47</a></p>
<p>This is orthogonal from the choice of FloodSub or GossipSub, but due to GossipSub’s more dynamic nature it is likely best combined with it.</p>
<h3 id="3-other-factors">3. Other factors</h3>
<p>Not a primary focus, but worth a look. Looking at the scaling model, there might be other easy wins to improve overall bandwidth consumption between full nodes. For example, can we reduce envelope size by a significant factor?</p>
<h2 id="track-3---accounting-and-user-run-nodes">Track 3 - Accounting and user-run nodes</h2>
<p>This is where we make sure the network isn’t fragile, become a true p2p app, get our users excited and engaged, and allow us to scale the network without creating an even bigger cluster.</p>
<p>To work in practice, this has a soft dependency on node discovery such as DNS based discovery (<a href="https://eips.ethereum.org/EIPS/eip-1459">https://eips.ethereum.org/EIPS/eip-1459</a>) or Discovery v5 (<a href="https://vac.dev/feasibility-discv5">https://vac.dev/feasibility-discv5</a>).</p>
<h3 id="1-adaptive-nodes-and-capabilities">1. Adaptive nodes and capabilities</h3>
<p>We want to make the gradation between light nodes, full nodes, storing (partial set of) historical messages, only acting for a specific shard, etc more flexible and explicit. This is required to identify and discover the nodes you want. See <a href="https://github.com/vacp2p/specs/issues/87">https://github.com/vacp2p/specs/issues/87</a></p>
<p>Depending on how the other tracks come together, this design should allow for a desktop node to identify as a full relaying node for some some app topic shard, but also express waku topic interest and retrieve historical messages itself.</p>
<p>E.g. Disc v5 can be used to supply node properties through ENR.</p>
<h3 id="2-accounting">2. Accounting</h3>
<p>This is based on a few principles:</p>
<ol>
<li>Some nodes contribute a lot more than other nodes in the network</li>
<li>We can account for the difference in contribution in some fashion</li>
<li>We want to incentivize nodes to tell the true, and be incentivized not to lie</li>
</ol>
<p>Accounting here is a stepping stone, where accounting is the raw data upon which some settlement later occurs. It can have various forms of granularity. See <a href="https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31">https://forum.vac.dev/t/accounting-for-resources-in-waku-and-beyond/31</a> for discussion.</p>
<p>We also note that in GossipSub, the mesh is bidrectional. Additionally, it doesn’t appears to be a high priority issue in terms of nodes misreporting. What is an issue is having people run full nodes in the first place. There are a few points to that. It has to be possible in the end-user UX, nodes have to be discovered, and it has to be profitable/visible that you are contributing. UX and discovery are out of scope for this work, whereas visibility/accounting is part of this scope. Settlement is a stretch goal here.</p>
<p>The general shape of the solution is inspired by the Swarm model, where we do accounting separate from settlement. It doesn’t require any specific proofs, but nodes are incentivized to tell the truth in the following way:</p>
<ol>
<li>Both full node and light node do accounting in a pairwise, local fashion</li>
<li>If a light node doesn’t ultimately pay or lie about reporting, they get disconnected (e.g.)</li>
<li>If a full node doesn’t provide its service the light node may pick another full node (e.g.)</li>
</ol>
<p>While accounting for individual resource usage is useful, for the ultimate end user experience we can ideally account for other things such as:</p>
<ul>
<li>end to end delivery</li>
<li>online time</li>
<li>completeness of storage</li>
</ul>
<p>This can be gradually enhanced and strengthened, for example with proofs, consistency checks, Quality of Service, reputation systems. See <a href="https://discuss.status.im/t/network-incentivisation-first-draft/1037">https://discuss.status.im/t/network-incentivisation-first-draft/1037</a> for one attempt to provide stronger guarantees with periodic consistency checks and a shared fund mechanism. And <a href="https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51">https://forum.vac.dev/t/incentivized-messaging-using-validity-proofs/51</a> for using validity proofs and removing liveness requirement for settlement.</p>
<p>All of this is optional at this stage, because our goal here is to improve the status quo for user run nodes. Accounting at this stage should be visible and correspond to the net benefit a node provides to another.</p>
<p>As a concrete example: a light node has some topic interest and cares about historical messages on some topic. A full node communicates envelopes as they come in, communicates their high availability (online time) and stores/forward stored messages. Both nodes have this information, and if they agree settlement (initially just a mock message) can be sending a payment to an address at some time interval / over some defined volume. See future sections for how this can be improved upon.</p>
<p>Also see below in section 4, using constructs such as eigentrust as a local reputation mechanism.</p>
<h3 id="3-relax-high-availability-requirement">3. Relax high availability requirement</h3>
<p>If we want desktop nodes to participate in the storing of historical messages, high availability is a problem. It is a problem for any node, especially if they lie about it, but assuming they are honest it is still an issue.</p>
<p>By being connected to multiple nodes, we can get an overlapping online window. Then these can be combined together to get consistency. This is obviously experimental and would need to be tested before being deployed, but if it works it’d be very useful.</p>
<p>Additionally or alternatively, instead of putting a high requirement on message availability, focus on detection of missing information. This likely requires re-thinking how we do data sync / replication.</p>
<h3 id="4-incentivize-light-and-full-nodes-to-tell-the-truth-policy-etc">4. Incentivize light and full nodes to tell the truth (policy, etc)</h3>
<p>In accounting phase it is largely assumed nodes are honest. What happens when they lie, and how do we incentivize them to be honest? In the case of Bittorrent this is done with tit-for-tat, however this is a different kind of relationship. What follows are some examples of how this can be done.</p>
<p>For light nodes:</p>
<ul>
<li>if they don’t, they get disconnected</li>
<li>prepayment (especially to “high value” nodes)</li>
</ul>
<p>For full nodes:</p>
<ul>
<li>multiple nodes reporting to agree, where truth becomes a shelling point</li>
<li>use eigentrust</li>
<li>staking for discovery visibility with slashing</li>
</ul>
<h3 id="5-settlement-poc">5. Settlement PoC</h3>
<p>Can be done after phase 2 if so desired. Basically integrate payments based on accounting and policy.</p>
<h1 id="out-of-scope">Out of scope</h1>
<ol>
<li>We assume the Status Base model requirements are accurate.</li>
<li>We assume Core will improve retention rates.</li>
<li>We assume the Stimbus production team will enable integration of nim-waku.</li>
<li>We assume Discovery mechanisms such as DNS and Discovery v5 will be worked on separately.</li>
<li>We assume Core will, at some point, provide an UX for integrating payment of services.</li>
<li>We assume the desktop client is sufficiently usable.</li>
<li>We assume Core and Infra will investigate ways of improving MaxPeers.</li>
</ol>
</div>
</div>
</section>
</main>
</div>
<footer class="footer bg-black flex flex-shrink-0 justify-center">
<div
class="
container
max-w-screen-xl
flex
sl:justify-between
lm:justify-start
p-5
pb-10
md:px-12 md:pt-5
lg:pt-10
"
>
<div class="logo mr-10 sm:mr-0 sm:w-2/12 lg:w-3/12">
<a href="/"
><img
src="/assets/img/logo.png"
alt="Vac logo"
class="w-9 h-11"
/></a>
</div>
<div
class="flex flex-col xm:flex-row xm:justify-between sm:w-10/12 lg:w-9/12"
>
<p
class="
hidden
ml:inline-block ml:mr-10
w-52
text-base text-white
opacity-75
lm:mr-32
"
>
VAC researches peer-to-peer, private, censorship resistant communication
</p>
<nav class="flex max-w-xs mr-0 xm:mr-5 l:mr-32 mb-5 sm:mb-0">
<div class="flex">
<div class="flex flex-col mr-5 sm:mr-10 sl:mr-14">
<p class="text-base text-white mb-5 lg:mb-4">Research</p>
<ul>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://status.im/" target="_blank" rel="noopener noreferrer"
>Log</a
>
</li>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://forum.vac.dev/" target="_blank" rel="noopener noreferrer"
>Forum</a
>
</li>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://status.im/" target="_blank" rel="noopener noreferrer"
>Media</a
>
</li>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://jobs.status.im/?search=Vac" target="_blank" rel="noopener noreferrer"
>Careers</a
>
</li>
</ul>
</div>
<div class="flex flex-col sl:mr-14">
<p class="text-base text-white mb-5 lg:mb-4 mr-5">Socials</p>
<ul>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://twitter.com/vacp2p" target="_blank" rel="noopener noreferrer"
>Twitter</a
>
</li>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://discord.gg/PQFdubGt6d" target="_blank" rel="noopener noreferrer"
>Discord</a
>
</li>
<li class="text-xxs lg:text-base text-white opacity-75 hover:opacity-50 mb-5">
<a href="https://t.me/vacp2p" target="_blank" rel="noopener noreferrer"
>Telegram</a
>
</li>
</ul>
</div>
</div></nav>
<div class="flex flex-col w-52">
<h3 class="text-base text-white mb-5">Signup for updates</h3>
<form action="" class="footer__form">
<div class="flex items-center w-full">
<input
class="
text-xs text-white
w-full
bg-black
border-b border-white
rounded-none
mr-2
focus:outline-none
placeholder-white placeholder-opacity-50
"
type="email"
placeholder="your email"
required
/>
<button
class="
h-10
w-10
bg-arrowWhite bg-auto bg-no-repeat bg-left
focus:outline-none
hover:opacity-50
"
type="submit"
></button>
</div>
</form>
<p
class="footer__confirm text-base italic text-white opacity-75 hidden"
>
You signed up! Check your e-mail
</p>
</div>
</div>
</div>
</footer>
<script src="/assets/js/main.js"></script>
<script src="/assets/js/smooth-scroll.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink