diff --git a/content/waku/updates/2024-06-20.md b/content/waku/updates/2024-06-20.md index 1c6c0e6b5..b5cac96b4 100644 --- a/content/waku/updates/2024-06-20.md +++ b/content/waku/updates/2024-06-20.md @@ -9,142 +9,141 @@ date: 2024-06-24 - [Store v3-beta - Message Hashes](https://github.com/waku-org/pm/issues/131) - achieved: - - PR bug fixes - - hash based query for outgoing messages https://github.com/status-im/status-go/pull/5217 + - [nwaku] PR bug fixes + - [chat] hash based query for outgoing messages https://github.com/status-im/status-go/pull/5217 - blockers: - - awaiting reviews - - unable to reproduce CI failures + - [chat] awaiting reviews + - [chat] unable to reproduce CI failures - [Store v3 - store synchronisation](https://github.com/waku-org/pm/issues/132) - achieved: - - waku-simulator testing - - Dogfooding and fixes for routine that checks for missing messages https://github.com/status-im/status-go/pull/5281 + - [nwaku] waku-simulator testing + - [chat] Dogfooding and fixes for routine that checks for missing messages https://github.com/status-im/status-go/pull/5281 - blockers: - - awaiting reviews - - unable to reproduce CI failures + - [chat] awaiting reviews + - [chat] unable to reproduce CI failures - [DOS protection for req-res protocols and metrics](https://github.com/waku-org/pm/issues/66) - achieved: - - Rate limit phase3: implemented per peer request rate checks - - BW metrics per shard: implemented per shard metric collection + - [nwaku] Rate limit phase3: implemented per peer request rate checks + - [nwaku] BW metrics per shard: implemented per shard metric collection - next: - - Rate limit phase3: Some polishing and test cases needed to finish - - Rate limit phase3: add rate limit metrics to dashboard - - BW metrics per shard: needs some tests and add section onto dashboard + - [nwaku] Rate limit phase3: Some polishing and test cases needed to finish + - [nwaku] Rate limit phase3: add rate limit metrics to dashboard + - [nwaku] BW metrics per shard: needs some tests and add section onto dashboard - [PostgreSQL Maintenance](https://github.com/waku-org/pm/issues/119) - achieved: - - enhance partition creation: https://github.com/waku-org/nwaku/issues/2783 - - validate that time retention policy works well: https://github.com/waku-org/nwaku/issues/2807 + - [nwaku] enhance partition creation: https://github.com/waku-org/nwaku/issues/2783 + - [nwaku] validate that time retention policy works well: https://github.com/waku-org/nwaku/issues/2807 ## [Milestone - Direct Message Reliability](https://github.com/waku-org/pm/milestone/29) - [Enable testing of direct messages](https://github.com/waku-org/pm/issues/176) - achieved: - - set up nightly tests & 1:1 messages tests https://github.com/status-im/status-cli-tests/pull/1 - - set up nightly tests for contact requests https://github.com/status-im/status-cli-tests/pull/3 - - CLI bugs and improvements for usage in nightly tests https://github.com/status-im/status-go/issues/5266 + - [chat] set up nightly tests & 1:1 messages tests https://github.com/status-im/status-cli-tests/pull/1 + - [chat] set up nightly tests for contact requests https://github.com/status-im/status-cli-tests/pull/3 + - [chat] CLI bugs and improvements for usage in nightly tests https://github.com/status-im/status-go/issues/5266 - next: - - set up nightly tests for group chats and community join requests - - investigate remaining tests failing + - [chat] set up nightly tests for group chats and community join requests + - [chat] investigate remaining tests failing - [Review connection management strategy and back-off and fix long disconnection issues](https://github.com/waku-org/pm/issues/177) - achieved: - - Pause peer connector if node is offline https://github.com/status-im/status-go/pull/5340, https://github.com/waku-org/go-waku/pull/1125 - - Fix usage of context for DiscV5 https://github.com/status-im/status-go/pull/5347 - - Fix in status-go to filter peers by shard - - Dogfood light client with peer exchange enabled + - [chat] Pause peer connector if node is offline https://github.com/status-im/status-go/pull/5340, https://github.com/waku-org/go-waku/pull/1125 + - [chat] Fix usage of context for DiscV5 https://github.com/status-im/status-go/pull/5347 + - [chat] Fix in status-go to filter peers by shard + - [chat] Dogfood light client with peer exchange enabled - next: - - Investigate / fix DiscV5 not finding valid peers in shards.test fleet - - make filter working along with peer exchange and debug connectivity issues with peers returned via peerExchange + - [chat] Investigate / fix DiscV5 not finding valid peers in shards.test fleet + - [chat] make filter working along with peer exchange and debug connectivity issues with peers returned via peerExchange - [Tooling: filter and light push protocols](https://github.com/waku-org/pm/issues/178) - achieved: - - lite-protocol-tester works on waku-simulator - - configurable stress conditions - - intensive stress test had been ran (45 nodes (relay + service nodes) + multiple publisher light client and a filter client receiver) - - ran with different conditions - - in co-op with Alberto analyzing topology and message flows + - [nwaku] lite-protocol-tester works on waku-simulator + - [nwaku] configurable stress conditions + - [nwaku] intensive stress test had been ran (45 nodes (relay + service nodes) + multiple publisher light client and a filter client receiver) + - [nwaku] ran with different conditions + - [nwaku] in co-op with Alberto analyzing topology and message flows - next: - - exclude docker limitations from the seen failed message deliveries + - [nwaku] exclude docker limitations from the seen failed message deliveries - [Telemetry: direct message reliability](https://github.com/waku-org/pm/issues/182) - achieved: - - implemented two different ways to log received message information. One based on libp2p observers https://github.com/waku-org/nwaku/pull/2800 which we won't use in practice. And one by adding the logs in a new nim-libp2p branch https://github.com/vacp2p/nim-libp2p/pull/1128 + - [nwaku] implemented two different ways to log received message information. One based on libp2p observers https://github.com/waku-org/nwaku/pull/2800 which we won't use in practice. And one by adding the logs in a new nim-libp2p branch https://github.com/vacp2p/nim-libp2p/pull/1128 - [Reliability Protocol for Relay](https://github.com/waku-org/pm/issues/184) - achieved: - - started to look at the current status-go implementation + - [nwaku] started to look at the current status-go implementation - next: - - carry on with the implementation in nwaku - https://github.com/waku-org/nwaku/issues/2819 + - [nwaku] carry on with the implementation in nwaku - https://github.com/waku-org/nwaku/issues/2819 - [Reliability Protocol for Resource-Restricted Clients](https://github.com/waku-org/pm/issues/186) - achieved: - - Improve filter subscription management in LightClient: various fixes wrt filter subscriptions and making lightNode work with peerExchange enabled https://github.com/status-im/status-go/pull/4665 - - dogfooding light client in desktop - - fixed issue found while dogfooing, remove stale subs and update ping interval https://github.com/waku-org/go-waku/pull/1119 - - improved peer management for light push protocol https://github.com/waku-org/js-waku/issues/2002 - - worked on updating js-waku to store v3 https://github.com/waku-org/js-waku/issues/2029 - - improving offline recoverability for Filter https://github.com/waku-org/js-waku/issues/2024 + - [chat] Improve filter subscription management in LightClient: various fixes wrt filter subscriptions and making lightNode work with peerExchange enabled https://github.com/status-im/status-go/pull/4665 + - [chat] dogfooding light client in desktop + - [chat] fixed issue found while dogfooing, remove stale subs and update ping interval https://github.com/waku-org/go-waku/pull/1119 + - [js-waku] improved peer management for light push protocol https://github.com/waku-org/js-waku/issues/2002 + - [js-waku] worked on updating js-waku to store v3 https://github.com/waku-org/js-waku/issues/2029 + - [js-waku] improving offline recoverability for Filter https://github.com/waku-org/js-waku/issues/2024 - next: - - continue with moving to store v3 https://github.com/waku-org/js-waku/issues/2029 - - continue with offline recoverability for Filter https://github.com/waku-org/js-waku/issues/2024 + - [js-waku] continue with moving to store v3 https://github.com/waku-org/js-waku/issues/2029 + - [js-waku] continue with offline recoverability for Filter https://github.com/waku-org/js-waku/issues/2024 - [User apps for large scale dogfooding](https://github.com/waku-org/pm/issues/188) - achieved: - - settled on the base implementation of dogfooding app https://github.com/waku-org/lab.waku.org/pull/68 + - [js-waku] settled on the base implementation of dogfooding app https://github.com/waku-org/lab.waku.org/pull/68 - next: - - provide front end for dogfooding app - - provide basic telemetry framework + - [js-waku] provide front end for dogfooding app + - [js-waku] provide basic telemetry framework - [Review MVDS usage and fail path](https://github.com/waku-org/pm/issues/189) - achieved: - - draft changes for reset MVDS epoch for online peers https://github.com/status-im/status-go/pull/5349 + - [chat] draft changes for reset MVDS epoch for online peers https://github.com/status-im/status-go/pull/5349 - next: - - continue MVDS review https://github.com/orgs/waku-org/projects/33 + - [chat] continue MVDS review https://github.com/orgs/waku-org/projects/33 ## [Milestone - End-to-end reliability protocol](https://github.com/waku-org/pm/milestone/30) - [End-to-end reliability protocol - PoC](https://github.com/waku-org/pm/issues/193) - achieved: - - continued gathering wide input on e2e reliability proposal: [https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293/](https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293/13) - - performed basic calculations to understand mathematical properties of proposed protocol, probabilistic model + - [nwaku] continued gathering wide input on e2e reliability proposal: [https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293/](https://forum.vac.dev/t/end-to-end-reliability-for-scalable-distributed-logs/293/13) + - [nwaku] performed basic calculations to understand mathematical properties of proposed protocol, probabilistic model - next: - - stakeholders sync on next steps - - set scope for POC implementation - - planning for a POC with bloom filter + - [nwaku] stakeholders sync on next steps + - [nwaku] set scope for POC implementation ## [Milestone - Static Sharding - dedicated shards](https://github.com/waku-org/pm/milestone/31) - [Sharding peer management and discovery hardening](https://github.com/waku-org/pm/issues/172) - achieved: - - start testing and investigating discv5 performance and possible issues https://github.com/waku-org/nwaku/issues/2810 - - started deep review of peer manager logic opened PR with small fix to be able to update peers' ENRs https://github.com/waku-org/nwaku/pull/2818 - - reproduced with DST team issues forming stable mesh https://github.com/waku-org/nwaku/issues/2780 and analyzed logs. Found that connections aren't being answered and started investigating a prospective problematic place in the code. + - [nwaku] start testing and investigating discv5 performance and possible issues https://github.com/waku-org/nwaku/issues/2810 + - [nwaku] started deep review of peer manager logic opened PR with small fix to be able to update peers' ENRs https://github.com/waku-org/nwaku/pull/2818 + - [nwaku] reproduced with DST team issues forming stable mesh https://github.com/waku-org/nwaku/issues/2780 and analyzed logs. Found that connections aren't being answered and started investigating a prospective problematic place in the code. - next: - - continue with discv5 and peer manager investigations - - continue investigating issuer forming a stable mesh + - [nwaku] continue with discv5 and peer manager investigations + - [nwaku] continue investigating issuer forming a stable mesh ## [Milestone - Scale 1:1 chat messages PoC](https://github.com/waku-org/pm/milestone/35) - [RLNv2 in nwaku](https://github.com/waku-org/pm/issues/204) - achieved: - - Published The Waku Simulator book: https://waku-org.github.io/waku-simulator/. It explains how to use waku-simulator to test different scenarios. - - completed RLNv2 test plan, waku-simulator updates + - [research] Published The Waku Simulator book: https://waku-org.github.io/waku-simulator/. It explains how to use waku-simulator to test different scenarios. + - [research] completed RLNv2 test plan, waku-simulator updates - next: - - Integrate waku spammer: https://github.com/waku-org/nwaku/pull/2821 - - Assist with rlnv2 testing. - - execute RLNv2 test plan + - [research] Integrate waku spammer: https://github.com/waku-org/nwaku/pull/2821 + - [research] Assist with rlnv2 testing. + - [research] execute RLNv2 test plan ## Other Work ### Research - [[Deliverable] Store Incentivisation (first iteration/POC)](https://github.com/waku-org/pm/issues/139) - - achieved: opened a PR for a PoC incentivization testing: https://github.com/waku-org/nwaku/pull/2816 - - next: start outlining a minimal specification for RLN-as-a-service + - [research] achieved: opened a PR for a PoC incentivization testing: https://github.com/waku-org/nwaku/pull/2816 + - [research] next: start outlining a minimal specification for RLN-as-a-service ### Reliability - achieved - - Increase store query pagesize https://github.com/status-im/status-go/pull/5341 - - Fix peer counter for statsu-desktop https://github.com/status-im/status-go/pull/5348 - - Change order of rpc text input https://github.com/status-im/status-desktop/pull/15169 + - [chat] Increase store query pagesize https://github.com/status-im/status-go/pull/5341 + - [chat] Fix peer counter for statsu-desktop https://github.com/status-im/status-go/pull/5348 + - [chat] Change order of rpc text input https://github.com/status-im/status-desktop/pull/15169