add waku weekly 20231127

content/waku/updates/2023-11-27.md

@@ -0,0 +1,147 @@
+---
+title: 2023-11-27 Waku Weekly
+tags:
+  - waku-updates
+date: 2023-11-27
+draft: false
+---
+## [Waku Network Can Support 10K Users](https://github.com/waku-org/pm/issues/12)
+
+- _achieved:_
+	- confirmed no unknown blockers from Waku's side to continue dogfooding in conversation with Status Communities
+	- continuing team-internal dogfooding of a test community using static sharding https://github.com/waku-org/pm/issues/97. See [dogfooding report](https://github.com/waku-org/pm/issues/97#issuecomment-1829050821)
+	- fleet ownership training: held session for stakeholders on responsibilities - https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741
+
+- _next:_
+	- continue dogfooding of Status Desktop with Status staging fleet with test community (https://github.com/waku-org/pm/issues/97)
+	- fix issue of store fleet not connecting to bootstrap fleet due to enr shards mismatch https://github.com/status-im/infra-shards/issues/23
+
+- _risks:_
+	- [Fleet Ownership doc](https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741) defines fleet _maintainer_ and _owner_. Status team yet to clarify who is the _fleet owner_ for Status Communities.
+	- QA by Status team to be planned on staging static sharding fleet; Waku team has done [internal dogfooding](https://github.com/waku-org/pm/issues/97) ([report](https://github.com/waku-org/pm/issues/97#issuecomment-1829050821)). Any change to the staging static sharding fleet should then be tested by QA before being deployed to prod (e.g. # of Postgres instances). Status has committed to this testing on 28Nov call.
+	- **Status team expressed will to deploy static sharding prod fleet and use it for all users: This is not recommended until proper QA is done on stagning static sharding fleet as it could impact other Status app activities**.
+	- Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev ([1](https://github.com/status-im/status-go/pull/4161), [2](https://github.com/status-im/status-go/pull/4094), [3](https://github.com/status-im/status-go/pull/4093)). See [status-go#4057](https://github.com/status-im/status-go/issues/4057) for remaining work. Mitigation by on-boarding Chat SDK team since November 2023 to drive effort.
+	- Dependency on Vac/DST to conclude ~1k nodes simulations; lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.
+
+### [Targeted dogfooding for Status Communities](https://github.com/waku-org/pm/issues/97)
+
+- _achieved_: fix issue of using default clusterID as 16 for shards fleet, dogfooding of status-desktop with shards fleet, debug issues in connection to fleet. See [dogfooding report](https://github.com/waku-org/pm/issues/97#issuecomment-1829050821)
+- _next_: continue dogfooding
+- _achieved_: fix pubsub topic used for store queries, logic for finding free port when initializing torrent service, add back changes related to default shard that were reverted before, store clusterID in app database, refactor mailserver cycle to not require having an active connection to a store node
+- _next_: dogfooding
+
+## [Waku Network can Support 1 Million Users](https://github.com/waku-org/pm/issues/83) - 2023-11-30
+
+- _next_: Pending DST simulations of 10k nodes gossipsub network.
+- _risks_:
+	- Dependency on Vac/DST to run 10k nodes simulations. Tracked under [`vac:dst:eng-10ktool`](https://roadmap.logos.co/tags/vac-updates).
+	- Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
+	- Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
+	- lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.
+
+  
+
+## [Waku Network Gen 0](https://github.com/waku-org/pm/issues/50) - 2023-12-01
+
+- _achieved_:
+	- Internal dogfooding of proto-network continues.
+	- js-waku work continues.
+	- nwaku optimization around peer management are underway.
+- _risks_:
+	- Usage of RLN in js-waku and dependency on a (centralized?) Web3Provider remains unclear as one needs to know the merkle tree state (on chain) to generate proofs.
+	- We are progressively moving a nwaku engineer to a solution engineer role we need to backfill the role.
+	- js-waku team is juggling between dev ex and gen 0 with only 2 engineers (3rd one just joined) so delivery in this client is likely to lag behind other clients.
+	- Uncertainty as to how RLN membership mechanism would hinder application adoption, if memberships need to be distributed or obtained by registration, if staking is necessary to prevent abuse, etc. Tracked with [#102](https://github.com/waku-org/pm/issues/102)
+
+### [4.1: Basic front end for node operator](https://github.com/waku-org/pm/issues/100)
+
+- _achieved_: follow ups and reduction of FE image;
+- _next_: double check with cors solved, minor improvements and fixes
+
+### [1.5: Launch and dogfood integrated public Waku Network MVP](https://github.com/waku-org/pm/issues/68)
+
+- _achieved_: come up with go-waku-compose to run a go-waku node in order to dogfood the public waku network
+- _next_: fix issues in go-waku-compose to support secure websockets and auto-fetch public IP
+
+### [1.4: Sharded peer management and discovery](https://github.com/waku-org/pm/issues/67)
+
+**[nwaku] [chore: add sharding information to peer storage](https://github.com/waku-org/nwaku/issues/2237)**
+
+- _achieved_: merged and closed
+
+**[nwaku] [chore: add sharding information to peer storage](https://github.com/waku-org/nwaku/issues/2237)**
+
+- _achieved_: updated peer storage to include ENR
+- _next_: review feedback cycle
+
+**[js-waku] [feat: Deprecate Named Sharding and Update Lightpush Client API](https://github.com/waku-org/js-waku/issues/1690)**
+
+- _achieved_: PR ready and currently in review-iteration phase ( #1697 )
+- _next_: to be merged
+
+**[js-waku] [feat: add new metadata protocol](https://github.com/waku-org/js-waku/issues/1652)** 
+
+- _next_: unblock this issue by merging #1697
+
+### [1.2: Autosharding for autoscaling](https://github.com/waku-org/pm/issues/65)
+
+**[nwaku] [chore: allow fetching cached messages by content or pubsub topic](https://github.com/waku-org/nwaku/issues/2201)**
+
+- _achieved_: PR merged DONE!
+
+### [2.1: Production testing of existing protocols](https://github.com/waku-org/pm/issues/49)
+
+**[nwaku] [feat: Migrate deployments to PostgreSQL](https://github.com/waku-org/nwaku/issues/1948)**
+
+- _achieved_: Preparing environment to stress one single database with multiple _Postgres_ clients writing and reading simultaneously.
+- _next_: Extend _Postgres_ benchmarking report from previous results and start analyzing the performance of `status.test` fleet where three nodes will use one single database.
+
+### [Presentation Readiness ](https://github.com/waku-org/pm/issues/95)
+
+**[js-waku] [feat: filter subscription API](https://github.com/waku-org/js-waku/issues/1683)**
+
+- _achieved_: PR opened and currently under review-iteration phase ( #1725 )
+- _next_: merge PR
+
+## Other Work
+### Enhancements
+
+**[nwaku] [chore(REST): adding to `/admin/v1/peers` response lightpush and filter v2 peer info](https://github.com/waku-org/nwaku/issues/2220)**
+
+- _achieved_: merged PR and closed issue
+
+**[js-noise] [feat: allow parameterization of handshakes](https://github.com/waku-org/js-noise/issues/48)**
+
+- _achieved_: implemented parameterization of DH, Cypher and Hash
+
+### Bugs
+
+**[nwaku] [bug: incomplete data sent or received log appearing when WSS is enabled](https://github.com/waku-org/nwaku/issues/2245)**
+
+- _achieved_: attempted reproducing, haven't gotten it to happen yet
+- _next_: succeed reproducing and fix
+
+**[nwaku] [bug: relay publish fails with 400 Bad Request when message contains meta field](https://github.com/waku-org/nwaku/issues/2214)**
+
+- _achieved_: analyzed issue and started implementing fixes
+- _next_: continue implementing the solution
+
+**[nwaku] [bug: relay push with malformed timestamp crashes nwaku](https://github.com/waku-org/nwaku/issues/2198)**
+
+- _achieved_: analyzed issue, found and implemented fix, raised PR in `nim-json-serialization` repo and implemented feedback. Merged fix and opened a PR in nwaku updating the dependency.
+- _next_: get nwaku PR reviewed y merge
+
+**[go-waku] [bug: `duplicate validator for topic` error when trying to re-subscribe to previously unsubscribed topic](https://github.com/waku-org/go-waku/issues/922)**
+
+- _achieved_: fix relay subscribe API issue causing failure in resubscription to a pubsubtopic, return appropriate errors in relay REST API
+  
+### Documentation
+
+**[docs.waku.org] [Advanced docs for js-waku](https://github.com/waku-org/docs.waku.org/issues/104)**
+
+- _achieved_: published manage filter guide, finished react docs
+- _next_: ECIES and symmetric encryption/decryption
+
+**[docs.waku.org] [Docs general improvement/incorporating feedback (2023)](https://github.com/waku-org/docs.waku.org/issues/102)**
+
+- _achieved_: added light mode toggle, updated the logos preset, added TWN guide, updated docs design and structure