<html><head><title>2023 September Monthly Waku Report</title><metacharSet="utf-8"/><metaname="viewport"content="width=device-width, initial-scale=1.0"/><metaproperty="og:title"content="2023 September Monthly Waku Report"/><metaproperty="og:description"content="Executive Summary § The month of September saw an agreement and solidification of the Waku roadmap which defines the process of launching the Waku Network as an independent piece of infrastructure the broader ecosystem can rely upon."/><metaproperty="og:image"content="https://roadmap.logos.co/static/og-image.png"/><metaproperty="og:width"content="1200"/><metaproperty="og:height"content="675"/><linkrel="icon"href="../../static/icon.png"/><metaname="description"content="Executive Summary § The month of September saw an agreement and solidification of the Waku roadmap which defines the process of launching the Waku Network as an independent piece of infrastructure the broader ecosystem can rely upon."/><metaname="generator"content="Quartz"/><linkrel="preconnect"href="https://fonts.googleapis.com"/><linkrel="preconnect"href="https://fonts.gstatic.com"/><linkhref="../../index.css"rel="stylesheet"type="text/css"spa-preserve/><linkhref="https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css"rel="stylesheet"type="text/css"spa-preserve/><linkhref="https://fonts.googleapis.com/css2?family=IBM Plex Mono&family=sans-serif:wght@400;700&family=sans-serif:ital,wght@0,400;0,600;1,400;1,600&display=swap"rel="stylesheet"type="text/css"spa-preserve/><scriptsrc="../../prescript.js"type="application/javascript"spa-preserve></script><scripttype="application/javascript"spa-preserve>constfetchData=fetch(`../../static/contentIndex.json`).then(data=>data.json())</script></head><bodydata-slug="waku/monthly-reports/2023-sept"><divid="quartz-root"class="page"><divid="quartz-body"><divclass="left sidebar"><h1class="page-title "><ahref="../..">Logos Collective Project Roadmaps</a></h1><divclass="spacer mobile-only"></div><divclass="search "><divid="search-icon"><p>Search</p><div></div><svgtabIndex="0"aria-labelledby="title desc"role="img"xmlns="http://www.w3.org/2000/svg"viewBox="0 0 19.9 19.7"><titleid="title">Search</title><descid="desc">Search</desc><gclass="search-path"fill="none"><pathstroke-linecap="square"d="M18.5 18.3l-5.4-5.4"></path><circlecx="8"cy="8"r="7"></circle></g></svg></div><divid="search-container"><divid="search-space"><inputautocomplete="off"id="search-bar"name="search"type="text"aria-label="Search for something"placeholder="Search for something"/><divid="results-container"></div></div></div></div><divclass="darkmode "><inputclass="toggle"id="darkmode-toggle"type="checkbox"tabIndex="-1"/><labelid="toggle-label-light"for="darkmode-toggle"tabIndex="-1"><svgxmlns="http://www.w3.org/2000/svg"xmlnsXlink="http://www.w3.org/1999/xlink"version="1.1"id="dayIcon"x="0px"y="0px"viewBox="0 0 35 35"style="enable-background:new 0 0 35 35;"xmlSpace="preserve"><title>Light mode</title><pathd="M6,17.5C6,16.672,5.328,16,4.5,16h-3C0.672,16,0,16.672,0,17.5S0.672,19,1.5,19h3C5.328,19,6,18.328,6,17.5zM7.5,26c-0.414,0-0.789,0.168-1.061,0.439l-2,2C4.168,28.711,4,29.086,4,29.5C4,30.328,4.671,31,5.5,31c0.414,0,0.789-0.168,1.06-0.44l2-2C8.832,28.289,9,27.914,9,27.5C9,26.672,8.329,26,7.5,26zM17.5,6C18.329,6,19,5.328,19,4.5v-3C19,0.672,18.329,0,17.5,0S16,0.672,16,1.5v3C16,5.328,16.671,6,17.5,6zM27.5,9c0.414,0,0.789-0.168,1.06-0.439l2-2C30.832,6.289,31,5.914,31,5.5C31,4.672,30.329,4,29.5,4c-0.414,0-0.789,0.168-1.061,0.44l-2,2C26.168,6.711,26,7.086,26,7.5C26,8.328,26.671,9,27.5,9zM6.439,8.561C6.711,8.832,7.086,9,7.5,9C8.328,9,9,8.328,9,7.5c0-0.414-0.168-0.789-0.439-1.061l-2-2C6.289,4.168,5.914,4,5.5,4C4.672,4,4,4.672,4,5.5c0,0.414,0.168,0.789,0.439,1.06L6.439,8.561zM33.5,16h-3c-0.828,0-1.5,0.672-1.5,1.5s0.672,1.5,1.5,1.5h3c0.828,0,1.5-0.672,1.5-1.5S34.328,16,33.5,16zM28.561,26.439C28.289,26.168,27.914,26,27.5,26c-0.828,0-1.5,0.672-1.5,1.5c0,0.414,0.168,0
<p>The month of September saw an agreement and solidification of the Waku roadmap which defines the process of launching the Waku Network as an independent piece of infrastructure the broader ecosystem can rely upon. Along with this, a revamp of the process in which work is labeled and tracked was performed which has an automated part and is generally more in line with the requests from the Insights team.</p>
<p>Work continues in the scaling and productionization efforts across the clients. Work previously done to enables PostgreSQL engine for <code>WAKU-STORE</code> in nwaku is being locally stress tested, further stress test using dev fleet is expected to be finalized next month. A Waku static sharding strategy is being integrated into Status to accommodate the first growth milestones of the re-release and adaptive sharding research and implementation is close behind.
Local simulation of <code>RLN-RELAY</code> has enabled the discovery of minor bugs which are fixed in the latest release of zerokit and nwaku and the study of the affect of RLN on performance.</p>
<p>Kurtosis as a platform was found to be insufficient for modeling a Waku network at the scale we wish, and the Vac team is pursuing an alternative strategy and writing up learnings from the boundaries were able to push.</p>
<p>Quality Assurance practices are scheduled throughout the next year to keep all client implementations up to a threshold of continuous quality.</p>
<p>The process to add native integration APIs to <code>nwaku</code> is underway such that the default native library moves from <code>go-waku</code> to <code>nwaku</code>.</p>
<p>A lot of work has been put into coalescing and finalizing the development tracking process that is in line with the Insight Reporting requirements, and Aaron’s addition to the team this month as pushed it over the edge to completion. Much of this has gone into automating the weekly reporting process via GitHub labels and comments on issues.</p>
<p>For tracking Waku maintains these Milestones in the <code>waku-org/pm</code> repo. Within each milestone description, you’ll find the corresponding Epics. Every Epic is distinctly labeled, and this label is affixed to each issue associated with that particular Epic. The labels are managed by the <code>labels.yaml</code> file located in the <code>waku-org/pm</code> repo.</p>
<p>Given the expansive nature of Waku and its various repositories working towards the milestones, the labels established in the <code>labels.yaml</code> file are replicated across each respective <code>waku-org</code> repo. This structure allows for seamless navigation, starting from top-level milestones down to the most granular issues.</p>
<p>Waku is broken out into the following four Milestones, with Epics associated with them:</p>
<ul>
<li><code>Waku Network Gen0</code></li>
<li><code>Waku Network Can Support 1MM Users</code></li>
<li><code>Quality Assurace Processes in Place</code></li>
<li><code>Support Many Platforms</code></li>
</ul>
<p>More details on the structure and progress of all Waku work can be tracked in their PM repository, specifically <ahref="https://github.com/waku-org/pm/milestones"class="external">the milestone page</a>. The following sections are highlighted updates on what happened this month.</p>
<p>The Waku Network RFC was created and <ahref="https://rfc.vac.dev/spec/64/"class="external">published on Vac</a> as RAW which details the ideas and architecture of the Waku Network. The next version of RLN was also <ahref="https://rfc.vac.dev/spec/58/"class="external">published on Vac</a> as RAW.</p>
<p>A benchmark of RLN was conducted and <ahref="https://github.com/waku-org/research/issues/23"class="external">the results</a> were discussed in a <ahref="https://minutes.logos.co/logos/logos-research-call-notes#september-27"class="external">Logos Research Call presentation</a> (See <ahref="https://github.com/waku-org/research/issues/23"class="external">waku-org/research#23</a> for details). The tl;dr is copied here for convenience:</p>
<li>Proof generation is constant-ish. 0.15 second for each proof</li>
<li>Proof verification is constant-ish, 0.012 seconds. In a network with 10k nodes and D=6 this would add an overhead delay of 0.06 seconds.</li>
<li>Gossipsub scoring drops connections from spammer peers, which acts as the punishment (instead of slashing). Validated in the simulation.</li>
<li>RLN doesn’t have any impact on memory consumption.</li>
</ul>
</blockquote>
<p>Based on these two specification publications and other associated work, efforts have begun to launch the first dev-testnet in time for the DevConnect event in November 2023.</p>
<h4id="waku-network-can-support-1mm-users"><ahref="https://github.com/waku-org/pm/issues/83"class="external">Waku Network Can Support 1MM Users</a><aaria-hidden="true"tabindex="-1"href="#waku-network-can-support-1mm-users"class="internal"> §</a></h4>
<p>Significant work was completed on the PostgreSQL integration and setup within <code>nwaku</code>, which supports the data retention and retrieval of Waku archival nodes. The implementation is currently being stress-tested to ensure production performance metrics are met.</p>
<p>The efforts in simulating a Waku Network of 10k users within a single shard continues and is tracked within <ahref="../../vac/dst/"class="internal"data-slug="vac/dst/">Vac DST Roadmap</a>. The performance of Wakurtosis (Kurtosis backend) was found to be insufficient for our requirements for scaling simulations. The creation of a Kubernetes orchestration tool, written in Python, has begun construction. This tool is heavily architected to mimic what Codex has created. It was chosen to reproduce this tooling in Python in order to increase usability and ease of maintenance/contribution as C# is a less known language within the org. The reasoning for this development can be tracked in <ahref="../../vac/dst/wakurtosis/vac/retrospective-rlog"class="internal"data-slug="vac/dst/wakurtosis/vac/retrospective-rlog">retrospective-rlog</a>.</p>
<p>The effort to understand a “professional Waku node operator” has begun and initial notes can be tracked within this <ahref="https://notes.status.im/node-operators-meeting"class="external">minutes doc</a>.</p>
<h4id="quality-assurance-processes-in-place"><ahref="https://github.com/waku-org/pm/issues/73"class="external">Quality Assurance Processes in Place</a><aaria-hidden="true"tabindex="-1"href="#quality-assurance-processes-in-place"class="internal"> §</a></h4>
<p>This milestone was created to ensure preparedness for the upcoming production client needs (specifically Waku Network Gen0 and Status Communities). A list of required processes in place was constructed and tasked out so that all implementations of Waku go through a standardized production release cycle.</p>
<p>This work is coordinating with the new <ahref="../../vac/dst/"class="internal"data-slug="vac/dst/">Vac DST</a> additions focused on testing rubrics for Logos Projects. This milestone is expected to be completed by Q1 2024.
Work has started to use the js-waku CI as an integration test suite for nwaku and go-waku. This test suite can now easily be run for either client as part of their release process.</p>
<h4id="support-many-platforms"><ahref="https://github.com/waku-org/pm/issues/42"class="external">Support Many Platforms</a><aaria-hidden="true"tabindex="-1"href="#support-many-platforms"class="internal"> §</a></h4>
<p>This is a large milestone created last month that tracks Waku’s “integration landscape” and attempting to ensure any developer seamlessly is able to integrate Waku.</p>
<p>It has started to list the work required for completion but more detail is needed to be fleshed out on prioritization and estimated resource needs. Currently, it is slotted for completion by April 2024.</p>
<p>Much of the work this month was fleshing out the available REST APIs of <code>nwaku</code>.</p>
<p>A <ahref="https://discord.com/channels/1110799176264056863/1111568123007606855/1149298157306531890"class="external">poll</a> was created to query what language priority we should have was gone. They’ll be published on socials next month to boost engagement and feedback. This poll will assist with priorities for this milestone.</p>
<h2id="perceived-changes-in-project-risk">Perceived Changes in Project Risk<aaria-hidden="true"tabindex="-1"href="#perceived-changes-in-project-risk"class="internal"> §</a></h2>
<ul>
<li>Waku doing most integration of Waku into <code>status-go</code> consumes a lot of Go-related developer resources.
<ul>
<li>a list of needed work is tracked <ahref="https://github.com/status-im/status-go/issues/4057"class="external">here</a></li>
</ul>
</li>
<li>There effort to convert <code>nwaku</code> to the main native integration client requires a large effort in the implementations of C-bindings in Nim and has some unknowns associated with it. Furthermore this additional effort and uncertainty doesn’t directly contribute to the current critical path of development.</li>
<li>The first main application of the milestone reorganization within the project has made the milestones associated with it clear, thus allowing the Waku Network MVP target setup to be tracked well</li>
<p>The insight team plans to further evaluate the value the reporting process implemented by Waku as it pertains to use within the other projects under Logos. It is expected that next month it will be finalized and ready for review by other teams to see if they’d like to adopt it.</p>
<p>One side effect of the automated reporting process is that the associate issue labels are already compatible with our data lake ingestion that was initiated by the Status project. This will allow us to create more useful dashboards and monitoring that take into account accurate development activity.</p>
