roadmap/waku/updates/2023-10-02.html

202 lines
99 KiB
HTML
Raw Normal View History

2023-10-02 22:07:33 +00:00
<!DOCTYPE html>
2023-10-05 21:22:31 +00:00
<html><head><title>2023-10-03 Waku Weekly</title><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta property="og:title" content="2023-10-03 Waku Weekly"/><meta property="og:description" content="waku-rust-bindings § feat: filterv2 support {E:RLN non-native SDKs} achieved: added support for unsubscribe, ping and unsubscribe_all filterv2 functions of go-waku c-bindings next: add support to subscribe nwaku § feat: Implement /admin Rest Api endpoint achieved: next: /admin rest endpoint feature is on PR review will be merged next week."/><meta property="og:image" content="https://roadmap.logos.co/static/og-image.png"/><meta property="og:width" content="1200"/><meta property="og:height" content="675"/><link rel="icon" href="../../static/icon.png"/><meta name="description" content="waku-rust-bindings § feat: filterv2 support {E:RLN non-native SDKs} achieved: added support for unsubscribe, ping and unsubscribe_all filterv2 functions of go-waku c-bindings next: add support to subscribe nwaku § feat: Implement /admin Rest Api endpoint achieved: next: /admin rest endpoint feature is on PR review will be merged next week."/><meta name="generator" content="Logos Roadmaps"/><link rel="preconnect" href="https://fonts.googleapis.com"/><link rel="preconnect" href="https://fonts.gstatic.com"/><link href="../../index.css" rel="stylesheet" type="text/css" spa-preserve/><link href="https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css" rel="stylesheet" type="text/css" spa-preserve/><link href="https://fonts.googleapis.com/css2?family=IBM Plex Mono&amp;family=sans-serif:wght@400;700&amp;family=sans-serif:ital,wght@0,400;0,600;1,400;1,600&amp;display=swap" rel="stylesheet" type="text/css" spa-preserve/><script src="../../prescript.js" type="application/javascript" spa-preserve></script><script type="application/javascript" spa-preserve>const fetchData = fetch(`../../static/contentIndex.json`).then(data => data.json())</script></head><body data-slug="waku/updates/2023-10-02"><div id="quartz-root" class="page"><div id="quartz-body"><div class="left sidebar"><h1 class="page-title"><a href="../..">Logos Collective Project Roadmaps</a></h1><div class="spacer mobile-only"></div><div class="search"><div id="search-icon"><p>Search</p><div></div><svg tabIndex="0" aria-labelledby="title desc" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 19.9 19.7"><title id="title">Search</title><desc id="desc">Search</desc><g class="search-path" fill="none"><path stroke-linecap="square" d="M18.5 18.3l-5.4-5.4"></path><circle cx="8" cy="8" r="7"></circle></g></svg></div><div id="search-container"><div id="search-space"><input autocomplete="off" id="search-bar" name="search" type="text" aria-label="Search for something" placeholder="Search for something"/><div id="results-container"></div></div></div></div><div class="darkmode"><input class="toggle" id="darkmode-toggle" type="checkbox" tabIndex="-1"/><label id="toggle-label-light" for="darkmode-toggle" tabIndex="-1"><svg xmlns="http://www.w3.org/2000/svg" xmlnsXlink="http://www.w3.org/1999/xlink" version="1.1" id="dayIcon" x="0px" y="0px" viewBox="0 0 35 35" style="enable-background:new 0 0 35 35;" xmlSpace="preserve"><title>Light mode</title><path d="M6,17.5C6,16.672,5.328,16,4.5,16h-3C0.672,16,0,16.672,0,17.5 S0.672,19,1.5,19h3C5.328,19,6,18.328,6,17.5z M7.5,26c-0.414,0-0.789,0.168-1.061,0.439l-2,2C4.168,28.711,4,29.086,4,29.5 C4,30.328,4.671,31,5.5,31c0.414,0,0.789-0.168,1.06-0.44l2-2C8.832,28.289,9,27.914,9,27.5C9,26.672,8.329,26,7.5,26z M17.5,6 C18.329,6,19,5.328,19,4.5v-3C19,0.672,18.329,0,17.5,0S16,0.672,16,1.5v3C16,5.328,16.671,6,17.5,6z M27.5,9 c0.414,0,0.789-0.168,1.06-0.439l2-2C30.832,6.289,31,5.914,31,5.5C31,4.672,30.329,4,29.5,4c-0.414,0-0.789,0.168-1.061,0.44 l-2,2C26.168,6.711,26,7.086,26,7.5C26,8.328,26.671,9,27.5,9z M6.439,8.561C6.711,8.832,7.086,9,7.5,9C8.328,9,9,8.328,9,7.5 c0-0.414-0.168-0.789-0.439-1.061l-2-2C6.289,4.168,5.914,4,5.5,4C4.672,4,4,4.672,4,5.5c0,0.414,0.168,0.789,0.439,1.06 L6.439,8.561z M33.5
2023-10-02 22:07:33 +00:00
<p><strong><a href="https://github.com/waku-org/waku-rust-bindings/issues/71" class="external">feat: filterv2 support</a></strong> {E:RLN non-native SDKs}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: added support for <code>unsubscribe</code>, <code>ping</code> and <code>unsubscribe_all</code> filterv2 functions of go-waku c-bindings</p>
</li>
<li>
<p><em>next</em>: add support to <code>subscribe</code></p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<hr/>
<h3 id="nwaku">nwaku<a aria-hidden="true" tabindex="-1" href="#nwaku" class="internal"> §</a></h3>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/2075" class="external">feat: Implement /admin Rest Api endpoint</a></strong></p>
<ul>
<li>
2023-10-05 21:22:31 +00:00
<p><em>achieved</em>:</p>
</li>
<li>
2023-10-02 22:07:33 +00:00
<p><em>next</em>: /admin rest endpoint feature is on PR review will be merged next week. Restructure openapi descriptions and producing swagger ui like live document of all rest interfaces.</p>
</li>
<li>
2023-10-05 21:22:31 +00:00
<p><em>blocking</em>: There are two build issues. libwaku cannot build on Fedora (RedHat) distros. Second, Abhi reported a build issue with wakunode2 - nim compiler crash under some circumstances.</p>
2023-10-02 22:07:33 +00:00
</li>
2023-10-05 21:22:31 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/2051" class="external">feat: RLN support for Nwaku-Compose</a></strong> {E:3.2: Basic DoS protection in production}</p>
<ul>
2023-10-02 22:07:33 +00:00
<li>
<p><em>achieved</em>: finished addressing feedback</p>
</li>
<li>
2023-10-05 21:22:31 +00:00
<p><em>next</em>: task is blocked until theres an easier method for users to register RLN credentials</p>
2023-10-02 22:07:33 +00:00
</li>
2023-10-05 21:22:31 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/1941" class="external">feat: Service peer selection on specific shards</a></strong> {E:1.4: Sharded peer management and discovery}</p>
<ul>
2023-10-02 22:07:33 +00:00
<li>
<p><em>achieved</em>: newly refactored STORE REST API handler that trigger discv5 peer search when needed.</p>
</li>
<li>
2023-10-05 21:22:31 +00:00
<p><em>next</em>: refactor other APIs</p>
2023-10-02 22:07:33 +00:00
</li>
2023-10-05 21:22:31 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/1888" class="external">PostgreSQL</a></strong> {E:2.1: Production testing of existing protocols}, {E:PostgreSQL}</p>
<ul>
2023-10-02 22:07:33 +00:00
<li>
<p><em>achieved</em>:</p>
</li>
<li>
<p>Better <code>dburl</code> parse that accepts host names with dashes and dots.</p>
</li>
<li>
<p>Properly set the compilation flag <code>-d:postgres</code> so <em>Docker</em> images are compiled with support to <em>Postgres</em> (with <code>libpq5</code> dependency.)</p>
</li>
<li>
<p>During the stress testing, I discovered that the max throughput seems not to be directly related to <em>Postgres</em>. If I make the code to ignore <em>Postgres</em> and return immediately a mocked response, then the throughput is even lower.</p>
</li>
<li>
2023-10-05 21:22:31 +00:00
<p><em>next</em>: Carry on with “select” performance analysis and analyze it directly from a <em>Store</em> client, rather than having <code>REST</code> &lt;-> <code>Store_Client</code> &lt;-> <code>Store_Server</code>. By ignoring the <code>REST</code> layer we will have a better insight into the actual <em>Store</em> protocol, as @jm-clius recommended to me some time ago.</p>
2023-10-02 22:07:33 +00:00
</li>
</ul>
2023-10-05 21:22:31 +00:00
<p><strong><a href="https://github.com/waku-org/nwaku/issues/1885" class="external">chore: add retention policy with GB or MB limitation</a></strong> {enhancement}, {E:PostgreSQL}</p>
<p>Added the new retention policy based on DB size.</p>
2023-10-02 22:07:33 +00:00
<p>Users can provide the size such as &lt;size_number>&lt;case_insenstive_gb_or_mb> ex. 30gb (which is also the default)</p>
2023-10-05 21:22:31 +00:00
<p><code>--store-message-retention-policy=size:25GB</code></p>
<p><code>--store-message-retention-policy=size:25gb</code></p>
<p><code>--store-message-retention-policy=size:250MB</code></p>
<p><code>--store-message-retention-policy=size:250mb</code></p>
2023-10-02 22:07:33 +00:00
<p>Test case also added.</p>
<p>Outdated messages/rows are deleted to suffice the size limit, with 20% size reduction upon overflowing.</p>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/1576" class="external">chore: update resolved enr ip when using <code>dns4-domain-name</code> flag</a></strong> {enhancement}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: addressed feedback and merged</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/1540" class="external">chore: improve test coverage on NetConfig generation</a></strong></p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: developed the new NetConfig test suite, raised PR, received and implemented feedback and merged.</p>
</li>
</ul>
<p><strong><a href="https://github.com/waku-org/nwaku/issues/1332" class="external">nwaku c-bindings (NodeJS + Python)</a></strong> {E:NodeJS Library}</p>
<ul>
<li>
<p><em>achieved</em>:</p>
</li>
<li>
<p>Added a simple cpp example to the main code. <a href="https://github.com/waku-org/nwaku/pull/2079" class="external">https://github.com/waku-org/nwaku/pull/2079</a>.</p>
</li>
<li>
<p>Submitted a PR where we start showing the doability of a Rust integration with the <code>libwaku</code>.</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<p>This PR is currently introducing the thread-safety enhancement of avoiding using global variables. Ideally, this should be in a separate PR. <a href="https://github.com/waku-org/nwaku/pull/2089" class="external">https://github.com/waku-org/nwaku/pull/2089</a>.</p>
<p>Notice that it was important to invest time in the Rust example so that we can carry on with the “callback” technique to exchange information between the host code (any) and the foreign code (Nim.)</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>next</em>: Separate the PR mentioned above and submit another one which only avoids using global variables but doesnt add the wip-Rust integration.</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<hr/>
<h3 id="js-waku">js-waku<a aria-hidden="true" tabindex="-1" href="#js-waku" class="internal"> §</a></h3>
<p><strong><a href="https://github.com/waku-org/js-waku/issues/1310" class="external">Static Sharding</a></strong> {E:Static sharding}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: allowing for multiple pubsub topics to be configured &amp; refactoring protocols to support</p>
</li>
<li>
<p><em>next</em>: enabling peer management to only dial relevant shards</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<hr/>
<h3 id="go-waku">go-waku<a aria-hidden="true" tabindex="-1" href="#go-waku" class="internal"> §</a></h3>
<p><strong><a href="https://github.com/waku-org/go-waku/issues/788" class="external">refactor: add user_data to c-bindings</a></strong> {E:RLN non-native SDKs}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: updated all the functions to include an additional <code>void* user_data</code> parameter</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/go-waku/issues/727" class="external">feat: discovery &amp; peer management for static shards</a></strong> {E:1.4: Sharded peer management and discovery}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: basic relay peer mgmt for static/auto sharding</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/go-waku/issues/680" class="external">feat: Service peer selection on specific shards</a></strong> {E:1.4: Sharded peer management and discovery}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: Peer selection updated to be based on pubsubTopic or contentTopic</p>
</li>
<li>
<p><em>next</em>: Update lightClient API to consider new peerSelection options</p>
</li>
2023-10-02 22:07:33 +00:00
</ul>
<p><strong><a href="https://github.com/waku-org/go-waku/issues/673" class="external">feat: Autosharding API for req-resp protocols</a></strong> {E:1.2: Autosharding for autoscaling}</p>
<ul>
2023-10-05 21:22:31 +00:00
<li>
<p><em>achieved</em>: Updated lightpush API for autosharding</p>
</li>
</ul>
<hr/></article></div><div class="right sidebar"><div class="graph"><h3>Graph View</h3><div class="graph-outer"><div id="graph-container" data-cfg="{&quot;drag&quot;:true,&quot;zoom&quot;:true,&quot;depth&quot;:1,&quot;scale&quot;:1.1,&quot;repelForce&quot;:0.5,&quot;centerForce&quot;:0.3,&quot;linkDistance&quot;:30,&quot;fontSize&quot;:0.6,&quot;opacityScale&quot;:1,&quot;showTags&quot;:true,&quot;removeTags&quot;:[]}"></div><svg version="1.1" id="global-graph-icon" xmlns="http://www.w3.org/2000/svg" xmlnsXlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 55 55" fill="currentColor" xmlSpace="preserve"><path d="M49,0c-3.309,0-6,2.691-6,6c0,1.035,0.263,2.009,0.726,2.86l-9.829,9.829C32.542,17.634,30.846,17,29,17
2023-10-02 22:07:33 +00:00
s-3.542,0.634-4.898,1.688l-7.669-7.669C16.785,10.424,17,9.74,17,9c0-2.206-1.794-4-4-4S9,6.794,9,9s1.794,4,4,4
c0.74,0,1.424-0.215,2.019-0.567l7.669,7.669C21.634,21.458,21,23.154,21,25s0.634,3.542,1.688,4.897L10.024,42.562
C8.958,41.595,7.549,41,6,41c-3.309,0-6,2.691-6,6s2.691,6,6,6s6-2.691,6-6c0-1.035-0.263-2.009-0.726-2.86l12.829-12.829
c1.106,0.86,2.44,1.436,3.898,1.619v10.16c-2.833,0.478-5,2.942-5,5.91c0,3.309,2.691,6,6,6s6-2.691,6-6c0-2.967-2.167-5.431-5-5.91
v-10.16c1.458-0.183,2.792-0.759,3.898-1.619l7.669,7.669C41.215,39.576,41,40.26,41,41c0,2.206,1.794,4,4,4s4-1.794,4-4
s-1.794-4-4-4c-0.74,0-1.424,0.215-2.019,0.567l-7.669-7.669C36.366,28.542,37,26.846,37,25s-0.634-3.542-1.688-4.897l9.665-9.665
C46.042,11.405,47.451,12,49,12c3.309,0,6-2.691,6-6S52.309,0,49,0z M11,9c0-1.103,0.897-2,2-2s2,0.897,2,2s-0.897,2-2,2
S11,10.103,11,9z M6,51c-2.206,0-4-1.794-4-4s1.794-4,4-4s4,1.794,4,4S8.206,51,6,51z M33,49c0,2.206-1.794,4-4,4s-4-1.794-4-4
s1.794-4,4-4S33,46.794,33,49z M29,31c-3.309,0-6-2.691-6-6s2.691-6,6-6s6,2.691,6,6S32.309,31,29,31z M47,41c0,1.103-0.897,2-2,2
s-2-0.897-2-2s0.897-2,2-2S47,39.897,47,41z M49,10c-2.206,0-4-1.794-4-4s1.794-4,4-4s4,1.794,4,4S51.206,10,49,10z"></path></svg></div><div id="global-graph-outer"><div id="global-graph-container" data-cfg="{&quot;drag&quot;:true,&quot;zoom&quot;:true,&quot;depth&quot;:-1,&quot;scale&quot;:0.9,&quot;repelForce&quot;:0.5,&quot;centerForce&quot;:0.3,&quot;linkDistance&quot;:30,&quot;fontSize&quot;:0.6,&quot;opacityScale&quot;:1,&quot;showTags&quot;:true,&quot;removeTags&quot;:[]}"></div></div></div><div class="toc desktop-only"><button type="button" id="toc"><h3>Table of Contents</h3><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="fold"><polyline points="6 9 12 15 18 9"></polyline></svg></button><div id="toc-content"><ul class="overflow"><li class="depth-0"><a href="#waku-rust-bindings" data-for="waku-rust-bindings">waku-rust-bindings</a></li><li class="depth-0"><a href="#nwaku" data-for="nwaku">nwaku</a></li><li class="depth-0"><a href="#js-waku" data-for="js-waku">js-waku</a></li><li class="depth-0"><a href="#go-waku" data-for="go-waku">go-waku</a></li></ul></div></div><div class="backlinks"><h3>Backlinks</h3><ul class="overflow"><li>No backlinks found</li></ul></div></div></div><footer><hr/><p>Created by Logos with <a href="https://quartz.jzhao.xyz/">Quartz v4.1.0</a>, © 2023</p><ul><li><a href="https://github.com/logos-co/roadmap">GitHub</a></li><li><a href="https://discord.com/invite/logos-state">Discord Community</a></li></ul></footer></div></body><script type="application/javascript">// quartz/components/scripts/quartz/components/scripts/callout.inline.ts
function toggleCallout() {
const outerBlock = this.parentElement;
outerBlock.classList.toggle(`is-collapsed`);
const collapsed = outerBlock.classList.contains(`is-collapsed`);
const height = collapsed ? this.scrollHeight : outerBlock.scrollHeight;
outerBlock.style.maxHeight = height + `px`;
let current = outerBlock;
let parent = outerBlock.parentElement;
while (parent) {
if (!parent.classList.contains(`callout`)) {
return;
}
const collapsed2 = parent.classList.contains(`is-collapsed`);
const height2 = collapsed2 ? parent.scrollHeight : parent.scrollHeight + current.scrollHeight;
parent.style.maxHeight = height2 + `px`;
current = parent;
parent = parent.parentElement;
}
}
function setupCallout() {
const collapsible = document.getElementsByClassName(
`callout is-collapsible`
);
for (const div of collapsible) {
const title = div.firstElementChild;
if (title) {
title.removeEventListener(`click`, toggleCallout);
title.addEventListener(`click`, toggleCallout);
const collapsed = div.classList.contains(`is-collapsed`);
const height = collapsed ? title.scrollHeight : div.scrollHeight;
div.style.maxHeight = height + `px`;
}
}
}
document.addEventListener(`nav`, setupCallout);
window.addEventListener(`resize`, setupCallout);
</script><script type="module">
import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.esm.min.mjs';
const darkMode = document.documentElement.getAttribute('saved-theme') === 'dark'
mermaid.initialize({
startOnLoad: false,
securityLevel: 'loose',
theme: darkMode ? 'dark' : 'default'
});
document.addEventListener('nav', async () => {
await mermaid.run({
querySelector: '.mermaid'
})
});
</script><script src="https://cdn.jsdelivr.net/npm/katex@0.16.7/dist/contrib/copy-tex.min.js" type="application/javascript"></script><script src="../../postscript.js" type="module"></script></html>