From a3905aa8fd4bbc31879d5ebedabcc71aaac0a0e5 Mon Sep 17 00:00:00 2001 From: chair28980 Date: Sun, 23 Jun 2024 16:01:10 -0700 Subject: [PATCH] create waku weekly update 2024-06-24 --- content/waku/updates/2024-06-20.md | 150 +++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 content/waku/updates/2024-06-20.md diff --git a/content/waku/updates/2024-06-20.md b/content/waku/updates/2024-06-20.md new file mode 100644 index 000000000..1c6c0e6b5 --- /dev/null +++ b/content/waku/updates/2024-06-20.md @@ -0,0 +1,150 @@ +--- +title: 2024-06-24 Waku Weekly +tags: + - waku-updates +date: 2024-06-24 +--- + +## [Milestone - Store Service Upgrade](https://github.com/waku-org/pm/milestone/28) + +- [Store v3-beta - Message Hashes](https://github.com/waku-org/pm/issues/131) + - achieved: + - PR bug fixes + - hash based query for outgoing messages https://github.com/status-im/status-go/pull/5217 + - blockers: + - awaiting reviews + - unable to reproduce CI failures + +- [Store v3 - store synchronisation](https://github.com/waku-org/pm/issues/132) + - achieved: + - waku-simulator testing + - Dogfooding and fixes for routine that checks for missing messages https://github.com/status-im/status-go/pull/5281 + - blockers: + - awaiting reviews + - unable to reproduce CI failures + +- [DOS protection for req-res protocols and metrics](https://github.com/waku-org/pm/issues/66) + - achieved: + - Rate limit phase3: implemented per peer request rate checks + - BW metrics per shard: implemented per shard metric collection + - next: + - Rate limit phase3: Some polishing and test cases needed to finish + - Rate limit phase3: add rate limit metrics to dashboard + - BW metrics per shard: needs some tests and add section onto dashboard + +- [PostgreSQL Maintenance](https://github.com/waku-org/pm/issues/119) + - achieved: + - enhance partition creation: https://github.com/waku-org/nwaku/issues/2783 + - validate that time retention policy works well: https://github.com/waku-org/nwaku/issues/2807 + +## [Milestone - Direct Message Reliability](https://github.com/waku-org/pm/milestone/29) + +- [Enable testing of direct messages](https://github.com/waku-org/pm/issues/176) + - achieved: + - set up nightly tests & 1:1 messages tests https://github.com/status-im/status-cli-tests/pull/1 + - set up nightly tests for contact requests https://github.com/status-im/status-cli-tests/pull/3 + - CLI bugs and improvements for usage in nightly tests https://github.com/status-im/status-go/issues/5266 + - next: + - set up nightly tests for group chats and community join requests + - investigate remaining tests failing + +- [Review connection management strategy and back-off and fix long disconnection issues](https://github.com/waku-org/pm/issues/177) + - achieved: + - Pause peer connector if node is offline https://github.com/status-im/status-go/pull/5340, https://github.com/waku-org/go-waku/pull/1125 + - Fix usage of context for DiscV5 https://github.com/status-im/status-go/pull/5347 + - Fix in status-go to filter peers by shard + - Dogfood light client with peer exchange enabled + - next: + - Investigate / fix DiscV5 not finding valid peers in shards.test fleet + - make filter working along with peer exchange and debug connectivity issues with peers returned via peerExchange + +- [Tooling: filter and light push protocols](https://github.com/waku-org/pm/issues/178) + - achieved: + - lite-protocol-tester works on waku-simulator + - configurable stress conditions + - intensive stress test had been ran (45 nodes (relay + service nodes) + multiple publisher light client and a filter client receiver) + - ran with different conditions + - in co-op with Alberto analyzing topology and message flows + - next: + - exclude docker limitations from the seen failed message deliveries + +- [Telemetry: direct message reliability](https://github.com/waku-org/pm/issues/182) + - achieved: + - implemented two different ways to log received message information. One based on libp2p observers https://github.com/waku-org/nwaku/pull/2800 which we won't use in practice. And one by adding the logs in a new nim-libp2p branch https://github.com/vacp2p/nim-libp2p/pull/1128 + +- [Reliability Protocol for Relay](https://github.com/waku-org/pm/issues/184) + - achieved: + - started to look at the current status-go implementation + - next: + - carry on with the implementation in nwaku - https://github.com/waku-org/nwaku/issues/2819 + +- [Reliability Protocol for Resource-Restricted Clients](https://github.com/waku-org/pm/issues/186) + - achieved: + - Improve filter subscription management in LightClient: various fixes wrt filter subscriptions and making lightNode work with peerExchange enabled https://github.com/status-im/status-go/pull/4665 + - dogfooding light client in desktop + - fixed issue found while dogfooing, remove stale subs and update ping interval https://github.com/waku-org/go-waku/pull/1119 + - improved peer management for light push protocol https://github.com/waku-org/js-waku/issues/2002 + - worked on updating js-waku to store v3 https://github.com/waku-org/js-waku/issues/2029 + - improving offline recoverability for Filter https://github.com/waku-org/js-waku/issues/2024 + - next: + - continue with moving to store v3 https://github.com/waku-org/js-waku/issues/2029 + - continue with offline recoverability for Filter https://github.com/waku-org/js-waku/issues/2024 + +- [User apps for large scale dogfooding](https://github.com/waku-org/pm/issues/188) + - achieved: + - settled on the base implementation of dogfooding app https://github.com/waku-org/lab.waku.org/pull/68 + - next: + - provide front end for dogfooding app + - provide basic telemetry framework + +- [Review MVDS usage and fail path](https://github.com/waku-org/pm/issues/189) + - achieved: + - draft changes for reset MVDS epoch for online peers https://github.com/status-im/status-go/pull/5349 + - next: + - continue MVDS review https://github.com/orgs/waku-org/projects/33 + +## [Milestone - End-to-end reliability protocol](https://github.com/waku-org/pm/milestone/30) + +- [End-to-end reliability protocol - PoC](https://github.com/waku-org/pm/issues/193) + - achieved: + - continued gathering wide input on e2e reliability proposal: [https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293/](https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293/13) + - performed basic calculations to understand mathematical properties of proposed protocol, probabilistic model + - next: + - stakeholders sync on next steps + - set scope for POC implementation + - planning for a POC with bloom filter + +## [Milestone - Static Sharding - dedicated shards](https://github.com/waku-org/pm/milestone/31) + +- [Sharding peer management and discovery hardening](https://github.com/waku-org/pm/issues/172) + - achieved: + - start testing and investigating discv5 performance and possible issues https://github.com/waku-org/nwaku/issues/2810 + - started deep review of peer manager logic opened PR with small fix to be able to update peers' ENRs https://github.com/waku-org/nwaku/pull/2818 + - reproduced with DST team issues forming stable mesh https://github.com/waku-org/nwaku/issues/2780 and analyzed logs. Found that connections aren't being answered and started investigating a prospective problematic place in the code. + - next: + - continue with discv5 and peer manager investigations + - continue investigating issuer forming a stable mesh + +## [Milestone - Scale 1:1 chat messages PoC](https://github.com/waku-org/pm/milestone/35) + +- [RLNv2 in nwaku](https://github.com/waku-org/pm/issues/204) + - achieved: + - Published The Waku Simulator book: https://waku-org.github.io/waku-simulator/. It explains how to use waku-simulator to test different scenarios. + - completed RLNv2 test plan, waku-simulator updates + - next: + - Integrate waku spammer: https://github.com/waku-org/nwaku/pull/2821 + - Assist with rlnv2 testing. + - execute RLNv2 test plan + +## Other Work + +### Research +- [[Deliverable] Store Incentivisation (first iteration/POC)](https://github.com/waku-org/pm/issues/139) + - achieved: opened a PR for a PoC incentivization testing: https://github.com/waku-org/nwaku/pull/2816 + - next: start outlining a minimal specification for RLN-as-a-service + +### Reliability +- achieved + - Increase store query pagesize https://github.com/status-im/status-go/pull/5341 + - Fix peer counter for statsu-desktop https://github.com/status-im/status-go/pull/5348 + - Change order of rpc text input https://github.com/status-im/status-desktop/pull/15169