2023-11-20 Waku weekly

Waku Network Can Support 10K Users

achieved:
- closed last PostgreSQL issue for Store scalability
- confirmed no unknown blockers from Waku's side to continue dogfooding in conversation with Status Communities
- started team-internal dogfooding of a test community using static sharding
- started fleet ownership handover process: published guidelines/list of responsibilities - https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741
next:
- continue dogfooding of Status Desktop with Status staging fleet with test community
- training session to conclude fleet ownership handover: https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741
risks:
- Dependency on Vac/DST to conclude ~1k nodes simulations.
- Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev (1, 2, 3). See status-go#4057 for remaining work. Mitigation by on-boarding Chat SDK lead on 6 Nov to drive effort.
- lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.
- lack of clarity in terms of Status fleet ownership, monitoring and maintenance, which is an integral part of the solution.

achieved: logout / login freeze, fix request on correct pubsub topic, and add missing shard information on community invite
next: dogfooding

achieved:
- Closed last Postgresql issue for basic Store scalability. See 10k milestone.
- Assisted DST in setting up initial tests with the ~1K tool. Currently still fine-tuning parameters, ensuring results are consistent, etc. for smaller configurations.
risks:
- Dependency on Vac/DST to run 10k nodes simulations. Tracked under vac:dst:eng-10ktool.
- Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
- Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
- lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.

achieved: derive pubsub topic from content topic in encoders/decoders when autosharding is specified
next: node config should specify static sharding or autosharding. implement autosharded topics in all req-resp protocols