From cd4768e6ae0fbccd7bc79c8a0f77069fc068345e Mon Sep 17 00:00:00 2001 From: chair <29414216+chair28980@users.noreply.github.com> Date: Sat, 20 Jul 2024 11:38:16 -0700 Subject: [PATCH] create waku weekly update 2024-07-22 (#89) --- content/waku/updates/2024-07-22.md | 122 +++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 content/waku/updates/2024-07-22.md diff --git a/content/waku/updates/2024-07-22.md b/content/waku/updates/2024-07-22.md new file mode 100644 index 000000000..ccefa7c8e --- /dev/null +++ b/content/waku/updates/2024-07-22.md @@ -0,0 +1,122 @@ +--- +title: 2024-07-22 Waku Weekly +tags: + - waku-updates +date: 2024-07-22 +--- + +## [Milestone - Store Service Upgrade](https://github.com/waku-org/pm/milestone/28) + +- [Store v3-beta - Message Hashes](https://github.com/waku-org/pm/issues/131) + - achieved: + - [research] Proof of concept of store testing with waku-simulator + - [chat] config to enable/disable storev3 confirmations for sent messages [feat: flag to enable/disable sent message store query confirmations](https://github.com/status-im/status-go/pull/5493) + - next: + - [research] Storev3 test plan and integrating tools for testing + - blockers: + - [research] Some complexities around merging/testing, currently handled by nwaku team + +- [DOS protection for req-res protocols and metrics](https://github.com/waku-org/pm/issues/66) + - achieved: + - [nwaku] BW metrics per shard: implemented per shard metric collection [feat: Proper bandwidth metrics per shard](https://github.com/waku-org/nwaku/issues/1945) + - [nwaku] Added dashboard panels for relay per shard traffic + - [nwaku] Added dashboard panels for non-relay protocols data traffic + - [nwaku] Added dashboard panels for non-relay protocols request rates + - [nwaku] Released [feat: Enforce service specific rate limits](https://github.com/waku-org/nwaku/issues/2032) + - Added per peer filtering of high users + - Load balancing compensation applied to token replenish + - Filter service specific limits for ping and subscribe per peer + +- [PostgreSQL Maintenance](https://github.com/waku-org/pm/issues/119) + - achieved: + - [nwaku] Improved and merged Sonda + - [nwaku] Merged store-v3 cursor fix [chore(archive): archive and drivers refactor](https://github.com/waku-org/nwaku/pull/2761) + - next: + - [nwaku] PostgreSQL query optimizations [chore: query performance in PostgreSQL by avoiding ordering in queries](https://github.com/waku-org/nwaku/issues/2902) + - [nwaku] PostgreSQL enhance retention policy: [bug: retention policy does not work when postgres db is over loaded](https://github.com/waku-org/nwaku/issues/2890) + - [nwaku] PostgreSQL enhance query performance [chore: improve postgres query performance](https://github.com/waku-org/nwaku/issues/2895) + +## [Milestone - Direct Message Reliability](https://github.com/waku-org/pm/milestone/29) + +- [Review connection management strategy and back-off and fix long disconnection issues](https://github.com/waku-org/pm/issues/177) + - achieved: + - [chat] [feat: flag to enable/disable missing message verification](https://github.com/status-im/status-go/pull/5497) + +- [Tooling: filter and light push protocols](https://github.com/waku-org/pm/issues/178) + - achieved: + - [chat] added support for 2 lightpush peers to be used for better reliability [feat: support for lightpush to use more than 1 peer](https://github.com/waku-org/go-waku/pull/1158) + - [chat] fix metadata protocol from disconnecting light-clients and only use clusterID and accept connections from lightclients [fix: for light node do not check for matching shards but only clusterID](https://github.com/waku-org/go-waku/pull/1154) + - next: + - [nwaku] Little enhancement for network connectivity, use bootstrap enr's to connect to any kind of network and detect shardings + - [nwaku] Auto select service peers and use them randomly for testing on both sides. + - [nwaku] run lite-protocol-tester on shards.staging + +- [Reliability Protocol for Relay](https://github.com/waku-org/pm/issues/184) + - achieved: + - [nwaku] started local sqlite registry implementation [feat: enhance reliability thanks to StoreV3](https://github.com/waku-org/nwaku/issues/2819) + - next: + - [nwaku] carry on with implementation [feat: enhance reliability thanks to StoreV3](https://github.com/waku-org/nwaku/issues/2819) + +- [Reliability Protocol for Resource-Restricted Clients](https://github.com/waku-org/pm/issues/186) + - achieved: + - [js-waku] improved renewal of peers for Filter [feat(filter): peer/subscription renewal with recurring Filter pings](https://github.com/waku-org/js-waku/pull/2052) + - [js-waku] check for missing messages from Filter subscriptions [feat: validate messages for individual filter nodes & perform renewals](https://github.com/waku-org/js-waku/pull/2057) + - [js-waku] peer renewal and keep alive improvements [feat: fix peer renewal, change Filter keep alive](https://github.com/waku-org/js-waku/pull/2065) + - next: + - [nwaku] Implementing: [feat: Enhance lightpush protocol error handling](https://github.com/waku-org/nwaku/issues/2722) + +- [Review MVDS usage and fail path](https://github.com/waku-org/pm/issues/189) + - achieved: + - [chat] status messages need for e2e reliability collected + - [chat] [fix_: delivered message should not send envelope sent signal](https://github.com/status-im/status-go/pull/5502) + +## [Milestone - End-to-end reliability protocol](https://github.com/waku-org/pm/milestone/30) + +- [End-to-end reliability protocol - PoC](https://github.com/waku-org/pm/issues/193) + - achieved: + - [research] implemented ACK handling, request missing messages, resend message + - next: + - [research] comprehensive tests for the poc + +## [Milestone - Static Sharding - dedicated shards](https://github.com/waku-org/pm/milestone/31) + +- [Sharding peer management and discovery hardening](https://github.com/waku-org/pm/issues/172) + - achieved: + - [research] review of discv5 both spec and nim implementation + - [research] compiled audit of discovery status quo and potential future strategies + - [nwaku] Creating versions for DST team to test discv5 and message reliability + - [nwaku] Proposing new solution to log received message info from nim-libp2p. Created custom version for DST team to use [chore: creating branch to test Waku's received messages](https://github.com/vacp2p/nim-libp2p/pull/1128) + - [nwaku] Started investigating and implementing improvements for connection management of bootstrap only nodes [bug: Node accepts connections when configured for just discovery](https://github.com/waku-org/nwaku/issues/2892) + - [nwaku] Improved logging [chore: improving logging under debugDiscv5 flag](https://github.com/waku-org/nwaku/pull/2899), [chore: logging content topic of spam messages](https://github.com/waku-org/nwaku/pull/2906) + - [nwaku] Investigated at depth discv5’s implementation [bug: discv5 not returning enough peers](https://github.com/waku-org/nwaku/issues/2810) + - next: + - [research] review other discovery methods and state of implementation in nim + - [research] review metrics collected and add more if needed + - [research] implement identified discv5 improvement strategies + - [nwaku] Get feedback on nim-libp2p’s logging solution + - [nwaku] Refactor code now that named shading is removed, and deprecate pubsub-topic configuration + - [nwaku] Continue discussing and analyzing discv5's performance and possible improvements + +## Other Work + +### Enhancements + +- achieved: + - [chat] [chore: rename shards.staging to status.staging](https://github.com/status-im/status-go/pull/5500) + - [chat] [chore: rename shards.staging to status.staging](https://github.com/status-im/status-desktop/pull/15510) + - [chat] [chore: bump go-libp2p](https://github.com/waku-org/go-waku/pull/1155) + - [chat] enable metadata protocol regardless of cluster-ID used + - [js-waku] [chore: enforce access modifiers](https://github.com/waku-org/js-waku/pull/2068) + - [js-waku] [chore: bump @waku peer dependencies](https://github.com/waku-org/js-waku/pull/2062) + - [js-waku] improve network options handling [chore: throw if more than one network config is passed](https://github.com/waku-org/js-waku/pull/2056) +- next: + - [nwaku] deploy release v0.31.0 [Prepare release 0.31.0](https://github.com/waku-org/nwaku/issues/2909) + +### Bugs + +- achieved: + - [chat] investigation of slowness to retrieve messages in storenode message counter + - [nwaku] Build nwaku image for Windows - [feature: support Windows 11](https://github.com/waku-org/nwaku/issues/2473) +- next: + - [nwaku] Manage stale peers dynamically with better reliability [fix: peer-exchange issue](https://github.com/waku-org/nwaku/pull/2889#event-13503654152) + - [nwaku] [bug: failed to retrieve peer info via peer exchange protocol](https://github.com/waku-org/nwaku/issues/2875)