roadmap/content/waku/updates/2023-11-06.md

title	tags	date	draft
2023-11-06 Waku weekly	waku-updates	2023-11-06	false

2023-11-06 Waku weekly

Waku Network Can Support 10K Users

• achieved:
  • further PostgreSQL optimisations nearing conclusion
  • implemented bridge to allow Status Community to move to static sharding with backwards compatibility towards default pubsub topic
  • solution for shared bootstrap nodes being filtered out in discv5 as more static shards are activated
  • ensured no unknown blockers from Waku's side to start dogfooding in conversation with Status Communities
• next:
  • continue integration of static sharding in status-go.
  • deploy bridge for backwards compatibility
  • dogfooding of Status Desktop with Status staging fleet. Will aim to create a small internal Waku community
• risks:
  • Dependency on Vac/DST to conclude ~1k nodes simulations.
  • Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev (1, 2, 3). See status-go#4057 for remaining work. Mitigation by on-boarding Chat SDK lead on 6 Nov to drive effort.
  • lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.
  • lack of clarity in terms of Status fleet ownership, monitoring and maintenance, which is an integral part of the solution.

Targeted dogfooding for Status Communities

• achieved: fix in status-go to use correct unprotected pubsub topic for community point of contacts, added pubsub topic bridge feature to go-waku, stop filtering out bootnodes on discovery, minimize noise on logs when selecting peers and not having peers available.
• next: deploy bridge between default pubsub topic and shard 32

Waku Network can Support 1 Million Users - 2023-11-30

• achieved:
  • See 10k milestone update for PostgreSQL status.
  • First version of the 10k-tool by DST is ready and is being tested with simulation running a small nim-libp2p/gossipsub binary.
• risks:
  • Dependency on Vac/DST to run 10k nodes simulations. Tracked under vac:dst:eng-10ktool.
  • Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
  • Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
  • lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.

PostgreSQL in service node: Further optimisations

[nwaku] PostgreSQL

• achieved: Optimize select/Store queries by adding prepared statements. PR
• next: Wrap up the Postgres optimizations. Summarize the performance comparison in a report.

Waku Network Gen 0 - 2023-12-01

• achieved:
  • Starting internal dogfooding of Waku Network: We have launched the proto-network and it is already possible to run a node in said network.
• risks:
  • Usage of RLN in js-waku and dependency on a (centralized?) Web3Provider remains unclear as one needs to know the merkle tree state (on chain) to generate proofs.
  • We are progressively moving a nwaku engineer to a solution engineer role we need to backfill the role.
  • js-waku team is juggling between dev ex and gen 0 with only 2 engineers (3rd one just joined) so delivery in this client is likely to lag behind other clients.
  • Uncertainty as to how RLN membership mechanism would hinder application adoption, if memberships need to be distributed or obtained by registration, if staking is necessary to prevent abuse, etc.

3.2: Basic DoS protection in production

[nwaku] feat: add rln to waku simulator instance

• achieved: integrated RLN, added Grafana dashboards, tested, merged and deployed

1.4: Sharded peer management and discovery

[go-waku] feat: Service peer selection on specific shards

• achieved: on-demand peer discovery for service peer selection and new relay shard subscriptions

2.3: Basic distributed Store services

[nwaku] feat: add new message_hash column to the archive protocol

• achieved: PR to support SQLite code to support messageHash attribute without interrupting the existing cursor-related functionality, id field stays for now. Skelton for sync in progress.
• next:
  • finalize the SQLite messageHash attribute and add a research page about it.
  • start a research page about the sync mechanism for nWaku, doing request/reply a PoC on the same.

Quality Assurance processes are in place - 2024-03-31

Support Many Platforms - 2024-04-30

Presentation Readiness

[js-waku] feat: better support for development

• achieved: by going through basic flow identified errors that can or cannot be ignored;
• next: working on improving log errors, aiming to complete in couple of days;

Other Work

Enhancements

[js-waku-examples] feat: make light-js example a proper debugging tool

• achieved: added peer dropdown, list of connected peers, and button for querying past messages using store
• next: will take on my first issue in js-waku

Bugs

[js-rln] bug: proof is not verified

• achieved: as per suggestion investigated if the roots are correct, seems found a fix;

Documentation

[docs.waku.org] Advanced docs for js-waku

• achieved: added createSubscription() docs in #128
• next: tackle @waku/sdk deprecated namespace #131, create filter management docs

[docs.waku.org] Node operator doc - cloud and advanced options

• achieved: updated nwaku config options in #128, added nwaku published MA option in #130

Chores

[nwaku] chore: bump vendor dependencies for 0.22.0

• achieved: updated dependencies, resolved conflicts, tested and raised PR
• next: get PR reviewed, implement feedback and merge

Ecosystem Development

• achieved:

  • Multiple leads from ETHLisbon.
  • Sync call with The Graph.
  • js-waku prioritizing.
  • Created Hackathon project with Waku at ETHLisbon.
  • Awesome-waku updates.

• next:

  • Review RLN docs readiness for launch at DevConnect.
  • Web3privacy event preparation.
  • Waku network soft-launch organisation.
  • Creating tutorial videos.
  • ETHStaker gathering for gaining some awareness about node operator onboarding.