From 7a4cd882b7fbe513a5aa008a3be5a57dd499925f Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Fri, 12 Jan 2024 16:40:44 +1100 Subject: [PATCH 01/13] Define 2024 Roadmap --- ROADMAP.md | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100644 ROADMAP.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..f6c75ef --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,132 @@ +# Waku Roadmap + +This live document acts as a high overview of the work likely to be tackled for Waku. + +This is not a commitment or planning document, it is aimed to be used as a discussion starter to prioritized and plan +work. +Actual planning and execution is done through GitHub issues and [milestones](https://github.com/waku-org/pm/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Amilestone). + +Definition and scoping strategies for a milestone are TBD. + +## Research Milestones + +WIP + +## Engineering Milestones + +### Node Bandwidth Management Mechanism + +Finish https://github.com/waku-org/pm/issues/66. + +### User Pays Own RLN Membership + +Provide a developer-ready API for a user to generate their own credentials and insert them. + +Ref: (A) https://github.com/waku-org/pm/issues/102 + +### Project Pays RLN Memberships for Users + +Provide a developer-ready API and software to enable developers to fund membership for users. +Some concession regarding the privacy of the developer/project infra may be acceptable (e.g. project hosts a REST API to +exchange private commitment). + +Ref: (B) https://github.com/waku-org/pm/issues/102 + +### Special User Pays RLN Memberships for Other Users + +This is a continuation of [Project Pays RLN Memberships for Users](#project-pays-rln-memberships-for-users). +However, requirements are stricter to preserve the privacy of the special user. + +Ref: (C) https://github.com/waku-org/pm/issues/102 + +### RLN Credentials Security and Multiple Devices + +Define how users should handle their credentials and provide the tooling for it. +Define whether RLN credentials should be shared between application or devices. +May also scope the work to enable sharing credentials between devices, or done as separate milestone. + +### Composing Waku Protocols to Improve Reliability + +Provide recommendation in a form of a library to compose the Waku protocols to minimize message loss at a protocol level. +This is both for relay and non-relay node. +The aim is to provide an opinionated library that use Waku in a generalized manner. +The library may not fit all use-case and can always be bypassed or forked. +Finally, the expectation is not 100% reliability but simply increased reliability for common scenarios. +Application level solution for more certitued are covered as part of [Minimum Viable Data Synchronization](#minimum-viable-data-synchronization). + +### Fault-Tolerant Usage of Web3 Providers + +Review the usage of Web3 Providers and provide a fault-tolerant strategy to ensure that The Waku Network cannot be DOS'd by Web3 Provider outage. +Strategies such as the software ability to fallback to alternative providers or usage of Ethereum Light Client and Portal network should be considered. + +### Waku Relay and Discv5 in a Hostile, Restricted or Subpar Online Network + +Proceed with a multi-client analysis of Waku Relay's performance in a somewhat online hostile or subpar network and the ability for a relay node to remain connected when said network: +- Hackathon with poor internet connectivity +- prevent incoming connections +- Usage of proxy or VPN to access Internet +- Mix of WAN/LAN configuration. +- Usage of IPV6 + +Note: Bluetooth, network mesh and internet outage are not in the scope of this milestone. + +### TLS-less Browser Connections + +Enable node operators to serve js-waku browser clients without acquiring a domain name. + +### Relay in The Browser + +Understand and progress on the viability of using Waku Relay in the browser. + +### NodeJS Library + +Provide a NodeJS package to enable user to run a relay node with Node. + +### Waku React Native + +Provide a functioning React Native library on both Android and iOS. +Likely using js-waku to enable multi-platform development (browser, iOS, Android). + +### JSON RPC Deprecation + +Deprecate nwaku and go-waku JSON RPC API and replace its usage with REST API across the repos (tests). + +### General Status Support + +Prioritize Status requirements, bugs or issues. + +### Developer Experience Feedback + +Expected to receive and action feedback from the usage of Waku libraries. +Feedback may be consolidated in a milestone. + +### Operator Feature Requests + +Not a milestone per se. +Review and potentially implement operator request feature requests. +Requests may come from Status Community operators. + +## Chat SDK Milestones + +### Static Sharding Integration in Status + +Complete static sharding integration in Status app and provide support for any issue encountered. + +### Status Community Scaling + +See https://github.com/vacp2p/research/issues/177 + +### Chat SDK PoC 1 + +TODO: decide on the scope of a first JS SDK for Chat. +This is likely to include accommodating the selected protocols to use the Waku Network Gen 0. + +### Minimum Viable Data Synchronization + +Provide a Minimum Viable Data Synchronization protocol that developer can use to ensure the delivery of messages in their protocol. +Note the [protocol](https://rfc.vac.dev/spec/2/) and a [Golang library](https://github.com/vacp2p/mvds) already exist. +The aim is to review and improve the protocol and library using Status feedback, implement the protocol for the browser and provide documentation. + +### Ethereum Address to Secure Channel + +Working MVP of https://rfc.vac.dev/spec/70/. From 9a22acfd0cf1b47c520723c029e474ff1ea10e35 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Wed, 24 Jan 2024 15:46:13 +1100 Subject: [PATCH 02/13] Include multiple client conclusion and refine milestones --- ROADMAP.md | 81 +++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 74 insertions(+), 7 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index f6c75ef..86abd2b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -18,6 +18,25 @@ WIP Finish https://github.com/waku-org/pm/issues/66. +### Bindings + +Provide functioning bindings over nwaku in the following environments: + +- NodeJS +- Rust (swap go-waku with nwaku) +- Golang + +### Serve Limited Content Topic (Store) + +Enable store service to only serve specific content topics. +Note that decentralized peer discovery for content topics is scheduled at a later stage, meaning that application +developers would likely need to pass or hardcode list of store node or an enrtree in their client. + +Risk: Need to cater for scenarios where a node connect to a store that does not support their content topic. +TODO: Should a small change to metadata protocol be planned? Should this be a research item instead? + +Ref: https://github.com/waku-org/pm/issues/64 + ### User Pays Own RLN Membership Provide a developer-ready API for a user to generate their own credentials and insert them. @@ -37,6 +56,9 @@ Ref: (B) https://github.com/waku-org/pm/issues/102 This is a continuation of [Project Pays RLN Memberships for Users](#project-pays-rln-memberships-for-users). However, requirements are stricter to preserve the privacy of the special user. +TODO: This could be postponed to a future date, depending on appetite, acceptable properties for Status Communities +usage and properties of [Project Pays RLN Memberships for Users](#project-pays-rln-memberships-for-users) solution. + Ref: (C) https://github.com/waku-org/pm/issues/102 ### RLN Credentials Security and Multiple Devices @@ -45,14 +67,20 @@ Define how users should handle their credentials and provide the tooling for it. Define whether RLN credentials should be shared between application or devices. May also scope the work to enable sharing credentials between devices, or done as separate milestone. +Note: Support needed from RLNP2P team. + ### Composing Waku Protocols to Improve Reliability Provide recommendation in a form of a library to compose the Waku protocols to minimize message loss at a protocol level. This is both for relay and non-relay node. -The aim is to provide an opinionated library that use Waku in a generalized manner. +The aim is to provide opinionated libraries that use Waku in a generalized manner. The library may not fit all use-case and can always be bypassed or forked. Finally, the expectation is not 100% reliability but simply increased reliability for common scenarios. -Application level solution for more certitued are covered as part of [Minimum Viable Data Synchronization](#minimum-viable-data-synchronization). +Application level solution for more certitude are covered as part of [Minimum Viable Data Synchronization](#minimum-viable-data-synchronization). + +This is part of the general SDK strategy and expected to be implemented in each language: +- JavaScript for Browser using js-waku +- Rust/Golang/JavaScript for Node using nwaku bindings. ### Fault-Tolerant Usage of Web3 Providers @@ -74,13 +102,13 @@ Note: Bluetooth, network mesh and internet outage are not in the scope of this m Enable node operators to serve js-waku browser clients without acquiring a domain name. +Note: WebRTC Direct and webtransport support needed in nim-libp2p and js-libp2p. + ### Relay in The Browser Understand and progress on the viability of using Waku Relay in the browser. -### NodeJS Library - -Provide a NodeJS package to enable user to run a relay node with Node. +Note: Support from js-libp2p maintainer needed. ### Waku React Native @@ -106,6 +134,15 @@ Not a milestone per se. Review and potentially implement operator request feature requests. Requests may come from Status Community operators. +### Ad hoc Network Formation + +Design and implement a protocol that uses TWN to negotiate the creation of an alternative small network that would enable lower-latency or higher bandwidth transfer. + +### Unscheduled + +- Serve only Content Topic Specific messages (filter, light push) +- Merge js-waku and NodeJS-nwaku APIs to enable usage of single SDK for JavaScript on all environments. + ## Chat SDK Milestones ### Static Sharding Integration in Status @@ -116,10 +153,28 @@ Complete static sharding integration in Status app and provide support for any i See https://github.com/vacp2p/research/issues/177 +### Use of nwaku in status-go + +Identify, plan and start the execution of replacing go-waku with nwaku in status-go. +Output for 2024 is a feasibility study is done, action plan is drafted and work has started. + +The action plan need to take in account the intent to provide Status chat protocol as libraries decoupled from status-go. +Yet, full dependency on said libraries should not be blocking. + ### Chat SDK PoC 1 -TODO: decide on the scope of a first JS SDK for Chat. -This is likely to include accommodating the selected protocols to use the Waku Network Gen 0. +Provide and consume JS library that enable a project to use Waku for chat purposes over TWN. +Exact scope to be define later. Likely either existing Status Chat 1:1 protocol or [Ethereum Address to Secure Channel](#ethereum-address-to-secure-channel). + +This library usage should be demonstrated in a web app a la wakuplay as part of dogfooding effort. + +### Extract 1 Status protocol from status-go + +This may or may not be the same item as [Chat SDK PoC 1](#chat-sdk-poc-1) as the consideration to be made are different +(market potential vs engineering feasibility within status-go context). + +This needs to be done with [Use of nwaku in status-go](#use-of-nwaku-in-status-go) with a bigger picture of status-go +future architecture in mind. ### Minimum Viable Data Synchronization @@ -130,3 +185,15 @@ The aim is to review and improve the protocol and library using Status feedback, ### Ethereum Address to Secure Channel Working MVP of https://rfc.vac.dev/spec/70/. + +### Unscheduled – 2025 Priority + +- Private group chat SDK +- Wakuplay usage to dogfood private group chat +- Support of Waku Objects and make it consumable + +## Other SDK Milestones + +### External Wallet Authentication and Transaction for Webapp + +Enable a web app to use Waku to request wallet signature for authentication and transaction purposes. From 6c916e4c778790fadb5a4acee29965ef227c83bc Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 12:10:37 +1100 Subject: [PATCH 03/13] Add PostgreSQL tooling around DB growth --- ROADMAP.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 86abd2b..5f39b2a 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -18,6 +18,11 @@ WIP Finish https://github.com/waku-org/pm/issues/66. +### PostgreSQL Maintenance + +Work on how to best handle PostgreSQL database growth and pruning is in progress and must completed to enable node operators +to handle database growth in an easy and non-disruptive manner. + ### Bindings Provide functioning bindings over nwaku in the following environments: @@ -142,6 +147,7 @@ Design and implement a protocol that uses TWN to negotiate the creation of an al - Serve only Content Topic Specific messages (filter, light push) - Merge js-waku and NodeJS-nwaku APIs to enable usage of single SDK for JavaScript on all environments. +- Review DB choices for embedded, and datacenter node. ## Chat SDK Milestones @@ -164,7 +170,7 @@ Yet, full dependency on said libraries should not be blocking. ### Chat SDK PoC 1 Provide and consume JS library that enable a project to use Waku for chat purposes over TWN. -Exact scope to be define later. Likely either existing Status Chat 1:1 protocol or [Ethereum Address to Secure Channel](#ethereum-address-to-secure-channel). +Exact scope to be defined later. Likely either existing Status Chat 1:1 protocol or [Ethereum Address to Secure Channel](#ethereum-address-to-secure-channel). This library usage should be demonstrated in a web app a la wakuplay as part of dogfooding effort. @@ -173,7 +179,7 @@ This library usage should be demonstrated in a web app a la wakuplay as part of This may or may not be the same item as [Chat SDK PoC 1](#chat-sdk-poc-1) as the consideration to be made are different (market potential vs engineering feasibility within status-go context). -This needs to be done with [Use of nwaku in status-go](#use-of-nwaku-in-status-go) with a bigger picture of status-go +This needs to be done with [use of nwaku in status-go](#use-of-nwaku-in-status-go) in mind with a bigger picture of status-go future architecture in mind. ### Minimum Viable Data Synchronization From 755e696099ae343c41fffde207d7c47ac421d1bc Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 21:34:26 +1100 Subject: [PATCH 04/13] Integrate Research milestones. --- ROADMAP.md | 307 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 298 insertions(+), 9 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 5f39b2a..a146004 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,16 +1,278 @@ # Waku Roadmap This live document acts as a high overview of the work likely to be tackled for Waku. - -This is not a commitment or planning document, it is aimed to be used as a discussion starter to prioritized and plan -work. Actual planning and execution is done through GitHub issues and [milestones](https://github.com/waku-org/pm/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Amilestone). -Definition and scoping strategies for a milestone are TBD. +MW means _man-week_: Effort time for an individual to complete the project. ## Research Milestones -WIP +### The Waku Whitepaper + +**Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols +**Research tracks:** All +**Estimated date of completion:** 2024Q4 +**Estimated effort:** +- research: 4 MW + +As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). +The document itself may source from one or more academic papers published and presented throughout the year. +Total length should be around 15 pages. + +### Waku RFC Review + +**Goal:** On completion, the set of RFCs for Waku will be simplified, reasonably up to date and accessible. +**Research tracks:** All +**Estimated date of completion:** 2024Q1 +**Estimated effort:** +- research: 3 MW +- docs/eco-dev: 1 MW + +Review our Waku RFC strategy, review the content of the most important RFCs, improve RFC indexing and ensure that +important RFCs are easily identified and accessible. + +2 outputs are expected to achieve this milestone: + +#### Review and implement RFC strategy + +**Ownership note**: this Epic is a collaboration between Waku Research (R/A) and Vac RFCs (C/I) + +Consult with stakeholders (Waku teams, Waku Community and Vac RFC team) to come up with an improved strategy for Waku RFCs to answer: + +1. Should Waku RFCs be in a separate Waku-owned repo? +2. What is the role of proof of concepts vs raw RFCs? +3. How should Waku RFCs be indexed? +4. How should Waku RFCs be presented to make important specs obvious and easy to find? +5. Should we continue with the same RFC categories (Standards, Informational and Best Current Practices)? +6. Should we continue with the same RFC tags (Waku core vs application)? +7. Should we continue with the same COSS lifecycle? Should we add a "stagnant" status a la EIPs? +8. How do we clarify the responsibility of the Vac team vs Waku team when it comes to RFCs? +9. How can external contributors contribute or propose improvements to Waku specifications? +10. Clarify intended scope of topics that can be covered in Waku RFCs, delineate categories clearly and revise the spec lifecycles. + +This item includes actioning the new proposed strategy, such as potentially moving Waku specs to a waku-org repo, +updating references to these in documentation, adding/removing tags and categories, writing up the proposed strategy +in contributor guidelines, etc. + +#### Review Waku RFCs + +**Ownership note**: this Epic is a collaboration between Waku Research (R/A) and Vac RFCs (C/I) + +Review the most important Waku RFCs: + +- review language for clarity +- ensure core specifications reflect the latest thinking around protocols and procedures +- consider marking some specifications as "stagnant" indicating that they're not being focused on +- deprecate RFCs that are no longer relevant + +### Store v3-beta - message hashes + +**Goal:** After this upgrade, the network will provide distributed and synchronised store services. +**Research tracks:** Message Reliability +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 5 MW +- nwaku +- js-waku +- docs/eco-dev: + +An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol: + +- introduces the concept of deterministic message hashes to index messages +- considers the Store as a key-value store +- allows for querying a list of keys (message hashes) from the Store +- allows for querying for the full message content (values) of a set of keys from the Store +- keeps all previous value-based filtering (e.g. content topic, timestamp) in place + +The [proposed PR](https://github.com/waku-org/waku-proto/pull/10/files) to simplify the Store protocol and use message +hashes as index/cursor, can be used as a starting point. + +### Store v3 - store synchronisation + +**Goal:** Upgrade the Store service capability in the network from a collection of local, unsynchronised, +semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation. +**Research tracks:** Message Reliability +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 9 MW +- nwaku: +- js-waku: +- go-waku: +- docs/eco-dev + +Building on Store v3-beta, this version of Store includes basic synchronisation between nodes. +This will probably include: + +- a protocol/heuristic to resume store services after an offline period +- a protocol/heuristic to periodically compare local key-value store with other nodes and find missing keys +- a protocol/heuristic to periodically download the messages (values) for missing keys from other store nodes + +Note that this can either be + +- (i) designing a new synchronisation protocol built into the Store protocol +- (ii) adapting existing synchronisation building blocks (e.g. a [Prolly Tree library](https://docs.canvas.xyz/blog/2023-05-04-merklizing-the-key-value-store.html)) into Store, or +- (iii) simply swapping the key-value store itself to a synchronised backend, such as [GossipLog](https://github.com/canvasxyz/canvas/tree/c1c208032b56d6b474996cb7da7bda5c55b69463/packages/gossiplog) + +IMO (iii) should be pursued as the preferred option, as far as possible. + +### Store Incentivisation (first iteration/POC) + +**Goal:** The network will provide proof of concept for incentivised store protocol +**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 5 weeks +- nwaku +- js-waku +- docs/eco-dev: + +A POC incentivisation mechanism that incorporates POC versions of the three Waku service incentivisation elements: + +- a price offer/negotiation mechanism +- a proof of payment system +- a local reputation mechanism to distinguish between "good" and "bad" store nodes + +> Note: Milestones in the incentivization domain may be significantly disrupted as the overall direction here is still being clarified re topics +> such as tokenomics, decisions about monetary vs reputational incentivisation, etc. Similarly, distributed reputation +> may soon form a very large part of this effort, but is not listed here as it's unclear where/if it fits the roadmap. + +### General Service Protocol Incentivisation (first iteration/POC) + +**Goal:** The network will provide proof of concept for incentivised service protocols +**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 5 weeks +- nwaku +- js-waku +- docs/eco-dev: + +This expands store incentivisation to other protocols: + +- consider the most reasonable incentivisation vectors for non-store service protocols, if they need incentivisation at all (filter, lightpush, peer-exchange) +- apply the POC incentivisation elements developed for Store to other service protocols where applicable, delineating alternative strategies where not. + +### Roadmap Towards Incentivisation on Mainnet + +**Goal:** Publish a breakdown clarifying the roadmap to push incentivization to mainnet. +**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 3 weeks + +The research and design necessary to come up with a roadmap to productionised incentivisation, including: + +1. evaluating the POC incentivisation mechanisms for fairness, security, exploitability and changing tactics where necessary +2. maturing our tokenomics, etc +3. establishing the (possible) role of a global reputation mechanism + +### Capability Service Discovery + +**Goal:** After this upgrade, the network will support decentralised service (e.g. store, filter) discovery on shards. +**Research tracks**: Secure Scaling, Restricted Run +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 6 MW +- ... + +Implement a strategy for Waku nodes to discover and maintain a healthy connectivity to the set of service peers +necessary for their configured applications. In the Waku Network nodes can opt to provide a range of services +(Store, Filter, etc.). Nodes can provide these services for one or more of the 8 pubsub topics comprising the Waku +Network. Any peer in the network trying to make use of these services should be able to easily find a service node +matching its desired (shard, service) capability vector. Various service discovery approaches is possible here. + +See https://github.com/waku-org/research/issues/74. + +### Content Service Discovery + +**Goal:** After this upgrade, the network will support service discovery on content topics +**Research tracks**: Secure Scaling, Restricted Run +**Estimated date of completion:** 2024Q4 + +**Estimated effort:** +- research 6 MW + +Implement a strategy for Waku nodes to discover service peers necessary for their configured applications based on the +content topics configured on those service nodes. In the Waku Network nodes can opt to provide a range of services +(Store, Filter, etc.). Nodes can provide these services for any set of content topic(s), allowing application-specific +service operators and nodes. Any peer in the network trying to make use of these services should be able to easily find +a service node matching its desired set of content topics. Note that the cardinality of all possible content topics +(infinite) may be too high for integration into existing DHTs/shared discovery methods. Alternative approaches could +include advertising on a separate pubsub topic, advertisements published on the configured content topics, etc. + +## Milestone: TWN - Rate-limiting Upgrade (RU) + +**Goal:** After this upgrade, the network will show maturity in how it limits rate, with better dimensioning for +bandwidth capping, resource-restricted options and a clear evaluation of what is needed to improve DoS protection. +**[Research tracks:](https://www.notion.so/Waku-Research-ac7941c0ece84dc6b2948ec02b3cb695?pvs=21)** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 30/09/2024 + +### RLN in resource-restricted clients + +**Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance. +**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 8 weeks +- js-waku... + +Enabling RLN proof generation and verification within light clients. +Different options are set out in [https://github.com/waku-org/research/issues/45](https://github.com/waku-org/research/issues/45). +Most recently, the promising option of deploying the entire RLN membership tree to a L2 as opposed to a L1 chain, +became the key focus of this effort. + +### RLNv2 + +**Goal:** Improved flexibility of the rate limit (from 1 msg/epoch to N msg/epoch), providing better dimensioning for bandwidth capping. +**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 4 weeks + +Moving from RLNv1 to RLNv2 to allow better bandwidth dimensioning in the network. This will allow a message allocation +per day per registered publisher, providing better statistical guarantees for network bandwidth usage. + +See: [https://github.com/waku-org/research/issues/22](https://github.com/waku-org/research/issues/22) for more + +### Maturing RLN variables/parameters revision (staking, contract/chain, token) + +**Goal:** A review of RLN security parameters and functionality in preparation for mainnet deployment. +**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 3 weeks + +Analyse RLN deployment in TWN Gen 0 and evaluate its DoS protection performance. + +- should staking be introduced, especially to improve resilience against adversarial membership registrations? +- should slashing be introduced or does the existing gossipsub scoring method provide enough protection? +- which chain or L2 should we target for memberships? +- if staking is introduced, which token should be used? +- do we need a combination of msg/sec and msg allocation/day rate limiting? + +### Unscheduled Milestones + +The following items are not yet scheduled, but may be prioritised in the first few months of 2024: + +- publish integrated peer and connection management strategy for all Waku protocols +- implement a distributed/global reputation mechanism +- pluggable validator strategy (see [https://github.com/waku-org/nwaku/issues/1700](https://github.com/waku-org/nwaku/issues/1700)) +- general Waku protocol security review and audit + +### Future Milestoness + +#### Decentralised bootstrapping + +Improve DNS-based discovery decentralisation by: + +1. including non-Waku nodes in the default bootstrap list/domain published by Waku +2. allowing non-Waku parties to easily create and publish their own lists, likely in the form of a discv5 DHT crawler that creates a DNS-discovery compatible Merkle tree of discovered nodes, signs and publishes these to a domain +3. using the DNS-discovery link tree functionality to link multiple published lists +4. allowing (and creating?) offline bootstrap lists maintained locally for each node + +See [https://github.com/waku-org/research/issues/69](https://github.com/waku-org/research/issues/69) for more + +### RLNv3 ## Engineering Milestones @@ -31,14 +293,18 @@ Provide functioning bindings over nwaku in the following environments: - Rust (swap go-waku with nwaku) - Golang +### Bindings - Mobile + +Verify and demonstrate that the Golang and Rust bindings work on mobile environments (Android and iOS). + ### Serve Limited Content Topic (Store) Enable store service to only serve specific content topics. Note that decentralized peer discovery for content topics is scheduled at a later stage, meaning that application developers would likely need to pass or hardcode list of store node or an enrtree in their client. -Risk: Need to cater for scenarios where a node connect to a store that does not support their content topic. -TODO: Should a small change to metadata protocol be planned? Should this be a research item instead? +Need to cater for scenarios where a node connect to a store that does not support their content topic, such as a +possible update of the metadata protocol. Ref: https://github.com/waku-org/pm/issues/64 @@ -143,7 +409,7 @@ Requests may come from Status Community operators. Design and implement a protocol that uses TWN to negotiate the creation of an alternative small network that would enable lower-latency or higher bandwidth transfer. -### Unscheduled +### Future Milestones - Serve only Content Topic Specific messages (filter, light push) - Merge js-waku and NodeJS-nwaku APIs to enable usage of single SDK for JavaScript on all environments. @@ -192,7 +458,7 @@ The aim is to review and improve the protocol and library using Status feedback, Working MVP of https://rfc.vac.dev/spec/70/. -### Unscheduled – 2025 Priority +### Future Milestones - Private group chat SDK - Wakuplay usage to dogfood private group chat @@ -203,3 +469,26 @@ Working MVP of https://rfc.vac.dev/spec/70/. ### External Wallet Authentication and Transaction for Webapp Enable a web app to use Waku to request wallet signature for authentication and transaction purposes. + +## Vac work Required + +**Ownership note**: these items are owned by Vac (R/A) and supported by Waku Research (C/I) + +1. Continue DST simulations: simulation of Waku network of 10K users, RLN simulations, etc. (DST) +2. Continue QA work: improve test coverage from a RFC and Status usage angle, including interoperability test run + between nwaku and go-waku, and once done, js-waku and nwaku. Considerations should be made to also include status-go (DST). +3. Simulate or model status-go behaviour based on the result of Waku large scale simulations. (DST) +4. Secure RLN credentials: for example in [CryptKeeper](https://github.com/CryptKeeperZK) (RLNP2P) +5. Support in improving RLN usage on resource-restricted devices (RLNP2P) +6. Secure way of delegating acquisition of memberships as described in [RLN Design Patterns](https://github.com/waku-org/pm/issues/102) (RLNP2P) +7. Fee structure for RLN memberships: come up with a comprehensive strategy for RLN memberships, perhaps following + the [NFT-like approach](https://forum.vac.dev/t/ens-like-nfts-for-rln/243). Strategy should consider cost structures + like Harberger tax, dev/node compensation and project sustainability, security, etc (RLNP2P, Tokenomics, Smart Contract) +8. RLN contract audited, reviewed and deployed to mainnet (RLNP2P, Smart Contract) +9. Investigate feasibility/risks for RLN on L2, recommend a specific L2 if any (RLNP2P, Tokenomics, Smart Contract) +10. Risk/attack analysis for RLN. How do we recover after an attack? (RLNP2P, Tokenomics) +11. Deliver a cryptographic protocol to enable setting up a secure channel for 1:1 and private group chat purposes from + an Ethereum address (ACZ). +12. Support in terms of cryptographic security, especially in regards of extending the Waku noise protocol if such a + need arises from Waku adoption (ACZ). + From 3f1ddbf68044cc1c0128b4058f014fa0b6755cb8 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 21:48:24 +1100 Subject: [PATCH 05/13] playing with format --- ROADMAP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ROADMAP.md b/ROADMAP.md index a146004..f69689e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -9,11 +9,13 @@ MW means _man-week_: Effort time for an individual to complete the project. ### The Waku Whitepaper +--- **Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols **Research tracks:** All **Estimated date of completion:** 2024Q4 **Estimated effort:** - research: 4 MW +--- As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). The document itself may source from one or more academic papers published and presented throughout the year. From f00d40d65c39154c2beadc2ccd2264e87f675b32 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 21:49:00 +1100 Subject: [PATCH 06/13] playing with format --- ROADMAP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index f69689e..0deda5e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,7 @@ MW means _man-week_: Effort time for an individual to complete the project. **Research tracks:** All **Estimated date of completion:** 2024Q4 **Estimated effort:** -- research: 4 MW + - research: 4 MW --- As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). From 07ab3318c936cd2dcfdcfe32f60af90c70f80df2 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 21:50:01 +1100 Subject: [PATCH 07/13] playing with format --- ROADMAP.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 0deda5e..29c4aab 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -9,13 +9,11 @@ MW means _man-week_: Effort time for an individual to complete the project. ### The Waku Whitepaper ---- -**Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols -**Research tracks:** All -**Estimated date of completion:** 2024Q4 -**Estimated effort:** - - research: 4 MW ---- + **Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols + **Research tracks:** All + **Estimated date of completion:** 2024Q4 + **Estimated effort:** + - research: 4 MW As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). The document itself may source from one or more academic papers published and presented throughout the year. From 7ac5dcba1f272e2665afeb2921653df7a95ea3b0 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 22:04:18 +1100 Subject: [PATCH 08/13] fix format --- ROADMAP.md | 153 ++++++++++++++++++++++++++--------------------------- 1 file changed, 76 insertions(+), 77 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 29c4aab..1562431 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -9,11 +9,11 @@ MW means _man-week_: Effort time for an individual to complete the project. ### The Waku Whitepaper - **Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols - **Research tracks:** All - **Estimated date of completion:** 2024Q4 - **Estimated effort:** - - research: 4 MW +- **Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols +- **Research tracks:** All +- **Estimated date of completion:** 2024Q4 +- **Estimated effort:** + - research: 4 MW As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). The document itself may source from one or more academic papers published and presented throughout the year. @@ -21,12 +21,12 @@ Total length should be around 15 pages. ### Waku RFC Review -**Goal:** On completion, the set of RFCs for Waku will be simplified, reasonably up to date and accessible. -**Research tracks:** All -**Estimated date of completion:** 2024Q1 -**Estimated effort:** -- research: 3 MW -- docs/eco-dev: 1 MW +- **Goal:** On completion, the set of RFCs for Waku will be simplified, reasonably up to date and accessible. +- **Research tracks:** All +- **Estimated date of completion:** 2024Q1 +- **Estimated effort:** + - research: 3 MW + - docs/eco-dev: 1 MW Review our Waku RFC strategy, review the content of the most important RFCs, improve RFC indexing and ensure that important RFCs are easily identified and accessible. @@ -67,14 +67,14 @@ Review the most important Waku RFCs: ### Store v3-beta - message hashes -**Goal:** After this upgrade, the network will provide distributed and synchronised store services. -**Research tracks:** Message Reliability -**Estimated date of completion:** 2024Q2 -**Estimated effort:** -- research: 5 MW -- nwaku -- js-waku -- docs/eco-dev: +- **Goal:** After this upgrade, the network will provide distributed and synchronised store services. +- **Research tracks:** Message Reliability +- **Estimated date of completion:** 2024Q2 +- **Estimated effort:** + - research: 5 MW + - nwaku + - js-waku + - docs/eco-dev: An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol: @@ -89,16 +89,16 @@ hashes as index/cursor, can be used as a starting point. ### Store v3 - store synchronisation -**Goal:** Upgrade the Store service capability in the network from a collection of local, unsynchronised, +- **Goal:** Upgrade the Store service capability in the network from a collection of local, unsynchronised, semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation. -**Research tracks:** Message Reliability -**Estimated date of completion:** 2024Q2 -**Estimated effort:** -- research: 9 MW -- nwaku: -- js-waku: -- go-waku: -- docs/eco-dev +- **Research tracks:** Message Reliability +- **Estimated date of completion:** 2024Q2 +- **Estimated effort:** + - research: 9 MW + - nwaku: + - js-waku: + - go-waku: + - docs/eco-dev Building on Store v3-beta, this version of Store includes basic synchronisation between nodes. This will probably include: @@ -117,14 +117,14 @@ IMO (iii) should be pursued as the preferred option, as far as possible. ### Store Incentivisation (first iteration/POC) -**Goal:** The network will provide proof of concept for incentivised store protocol -**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization -**Estimated date of completion:** 2024Q2 -**Estimated effort:** -- research: 5 weeks -- nwaku -- js-waku -- docs/eco-dev: +- **Goal:** The network will provide proof of concept for incentivised store protocol +- **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +- **Estimated date of completion:** 2024Q2 +- **Estimated effort:** + - research: 5 weeks + - nwaku + - js-waku + - docs/eco-dev: A POC incentivisation mechanism that incorporates POC versions of the three Waku service incentivisation elements: @@ -138,14 +138,14 @@ A POC incentivisation mechanism that incorporates POC versions of the three Waku ### General Service Protocol Incentivisation (first iteration/POC) -**Goal:** The network will provide proof of concept for incentivised service protocols -**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization -**Estimated date of completion:** 2024Q2 -**Estimated effort:** -- research: 5 weeks -- nwaku -- js-waku -- docs/eco-dev: +- **Goal:** The network will provide proof of concept for incentivised service protocols +- **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +- **Estimated date of completion:** 2024Q2 +- **Estimated effort:** + - research: 5 weeks + - nwaku + - js-waku + - docs/eco-dev: This expands store incentivisation to other protocols: @@ -154,11 +154,11 @@ This expands store incentivisation to other protocols: ### Roadmap Towards Incentivisation on Mainnet -**Goal:** Publish a breakdown clarifying the roadmap to push incentivization to mainnet. -**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization -**Estimated date of completion:** 2024Q3 -**Estimated effort:** -- research: 3 weeks +- **Goal:** Publish a breakdown clarifying the roadmap to push incentivization to mainnet. +- **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +- **Estimated date of completion:** 2024Q3 +- **Estimated effort:** + - research: 3 weeks The research and design necessary to come up with a roadmap to productionised incentivisation, including: @@ -168,12 +168,12 @@ The research and design necessary to come up with a roadmap to productionised in ### Capability Service Discovery -**Goal:** After this upgrade, the network will support decentralised service (e.g. store, filter) discovery on shards. -**Research tracks**: Secure Scaling, Restricted Run -**Estimated date of completion:** 2024Q2 -**Estimated effort:** -- research: 6 MW -- ... +- **Goal:** After this upgrade, the network will support decentralised service (e.g. store, filter) discovery on shards. +- **Research tracks**: Secure Scaling, Restricted Run +- **Estimated date of completion:** 2024Q2 +- **Estimated effort:** + - research: 6 MW + - ... Implement a strategy for Waku nodes to discover and maintain a healthy connectivity to the set of service peers necessary for their configured applications. In the Waku Network nodes can opt to provide a range of services @@ -185,12 +185,11 @@ See https://github.com/waku-org/research/issues/74. ### Content Service Discovery -**Goal:** After this upgrade, the network will support service discovery on content topics -**Research tracks**: Secure Scaling, Restricted Run -**Estimated date of completion:** 2024Q4 - -**Estimated effort:** -- research 6 MW +- **Goal:** After this upgrade, the network will support service discovery on content topics +- **Research tracks**: Secure Scaling, Restricted Run +- **Estimated date of completion:** 2024Q4 +- **Estimated effort:** + - research 6 MW Implement a strategy for Waku nodes to discover service peers necessary for their configured applications based on the content topics configured on those service nodes. In the Waku Network nodes can opt to provide a range of services @@ -209,12 +208,12 @@ bandwidth capping, resource-restricted options and a clear evaluation of what is ### RLN in resource-restricted clients -**Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance. -**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting -**Estimated date of completion:** 2024Q3 -**Estimated effort:** -- research: 8 weeks -- js-waku... +- **Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance. +- **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +- **Estimated date of completion:** 2024Q3 +- **Estimated effort:** + - research: 8 weeks + - js-waku... Enabling RLN proof generation and verification within light clients. Different options are set out in [https://github.com/waku-org/research/issues/45](https://github.com/waku-org/research/issues/45). @@ -223,11 +222,11 @@ became the key focus of this effort. ### RLNv2 -**Goal:** Improved flexibility of the rate limit (from 1 msg/epoch to N msg/epoch), providing better dimensioning for bandwidth capping. -**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting -**Estimated date of completion:** 2024Q3 -**Estimated effort:** -- research: 4 weeks +- **Goal:** Improved flexibility of the rate limit (from 1 msg/epoch to N msg/epoch), providing better dimensioning for bandwidth capping. +- **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +- **Estimated date of completion:** 2024Q3 +- **Estimated effort:** + - research: 4 weeks Moving from RLNv1 to RLNv2 to allow better bandwidth dimensioning in the network. This will allow a message allocation per day per registered publisher, providing better statistical guarantees for network bandwidth usage. @@ -236,11 +235,11 @@ See: [https://github.com/waku-org/research/issues/22](https://github.com/waku-or ### Maturing RLN variables/parameters revision (staking, contract/chain, token) -**Goal:** A review of RLN security parameters and functionality in preparation for mainnet deployment. -**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting -**Estimated date of completion:** 2024Q3 -**Estimated effort:** -- research: 3 weeks +- **Goal:** A review of RLN security parameters and functionality in preparation for mainnet deployment. +- **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +- **Estimated date of completion:** 2024Q3 +- **Estimated effort:** + - research: 3 weeks Analyse RLN deployment in TWN Gen 0 and evaluate its DoS protection performance. From f2c90befc25743b28ef196dbeb756400f7d288d4 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 22:34:58 +1100 Subject: [PATCH 09/13] Remove old section --- ROADMAP.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 1562431..be294f8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -65,7 +65,7 @@ Review the most important Waku RFCs: - consider marking some specifications as "stagnant" indicating that they're not being focused on - deprecate RFCs that are no longer relevant -### Store v3-beta - message hashes +### Store v3-beta - Message Hashes - **Goal:** After this upgrade, the network will provide distributed and synchronised store services. - **Research tracks:** Message Reliability @@ -199,13 +199,6 @@ a service node matching its desired set of content topics. Note that the cardina (infinite) may be too high for integration into existing DHTs/shared discovery methods. Alternative approaches could include advertising on a separate pubsub topic, advertisements published on the configured content topics, etc. -## Milestone: TWN - Rate-limiting Upgrade (RU) - -**Goal:** After this upgrade, the network will show maturity in how it limits rate, with better dimensioning for -bandwidth capping, resource-restricted options and a clear evaluation of what is needed to improve DoS protection. -**[Research tracks:](https://www.notion.so/Waku-Research-ac7941c0ece84dc6b2948ec02b3cb695?pvs=21)** Secure Scaling, Restricted Run, Rate Limiting -**Estimated date of completion:** 30/09/2024 - ### RLN in resource-restricted clients - **Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance. From e8897a234357a20a8b92be387634e45bfe920783 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 22:56:36 +1100 Subject: [PATCH 10/13] vac update --- ROADMAP.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index be294f8..d95dae5 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -277,7 +277,7 @@ Finish https://github.com/waku-org/pm/issues/66. Work on how to best handle PostgreSQL database growth and pruning is in progress and must completed to enable node operators to handle database growth in an easy and non-disruptive manner. -### Bindings +### Bindings (NodeJS, Rust, Golang) Provide functioning bindings over nwaku in the following environments: @@ -479,8 +479,10 @@ Enable a web app to use Waku to request wallet signature for authentication and 8. RLN contract audited, reviewed and deployed to mainnet (RLNP2P, Smart Contract) 9. Investigate feasibility/risks for RLN on L2, recommend a specific L2 if any (RLNP2P, Tokenomics, Smart Contract) 10. Risk/attack analysis for RLN. How do we recover after an attack? (RLNP2P, Tokenomics) -11. Deliver a cryptographic protocol to enable setting up a secure channel for 1:1 and private group chat purposes from +11. Deliver WebRTC Direct support in nim-libp2p (P2P) +12. Deliver WebTransport support in nim-libp2p (P2P) +13. Deliver a cryptographic protocol to enable setting up a secure channel for 1:1 and private group chat purposes from an Ethereum address (ACZ). -12. Support in terms of cryptographic security, especially in regards of extending the Waku noise protocol if such a +14. Support in terms of cryptographic security, especially in regards of extending the Waku noise protocol if such a need arises from Waku adoption (ACZ). From fffa450b0d20c3aac2479106e8b2217706f68ae1 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Tue, 30 Jan 2024 15:51:10 +1100 Subject: [PATCH 11/13] Add deliverables information --- ROADMAP.md | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index d95dae5..9dafa48 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,15 +1,27 @@ -# Waku Roadmap +# Waku Roadmap 2024 This live document acts as a high overview of the work likely to be tackled for Waku. Actual planning and execution is done through GitHub issues and [milestones](https://github.com/waku-org/pm/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Amilestone). MW means _man-week_: Effort time for an individual to complete the project. +## Deliverable + +Waku commits to 3 deliverables (WIP): + +1. RLN membership growth +2. Status Support +3. Progress on Censorship-resistance, Privacy and Sustainability. + +(~2) expresses that while this deliverable is expected to be needed by Status, it's usage may not happen in 2024 and is +not part of the list of requirements for Status in 2024. + ## Research Milestones ### The Waku Whitepaper - **Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols +- **Deliverables**: (3) - **Research tracks:** All - **Estimated date of completion:** 2024Q4 - **Estimated effort:** @@ -22,6 +34,7 @@ Total length should be around 15 pages. ### Waku RFC Review - **Goal:** On completion, the set of RFCs for Waku will be simplified, reasonably up to date and accessible. +- **Deliverables**: (3) - **Research tracks:** All - **Estimated date of completion:** 2024Q1 - **Estimated effort:** @@ -68,6 +81,7 @@ Review the most important Waku RFCs: ### Store v3-beta - Message Hashes - **Goal:** After this upgrade, the network will provide distributed and synchronised store services. +- **Deliverables**: (2), (3) - **Research tracks:** Message Reliability - **Estimated date of completion:** 2024Q2 - **Estimated effort:** @@ -91,6 +105,7 @@ hashes as index/cursor, can be used as a starting point. - **Goal:** Upgrade the Store service capability in the network from a collection of local, unsynchronised, semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation. +- **Deliverables**: (2), (3) - **Research tracks:** Message Reliability - **Estimated date of completion:** 2024Q2 - **Estimated effort:** @@ -118,6 +133,7 @@ IMO (iii) should be pursued as the preferred option, as far as possible. ### Store Incentivisation (first iteration/POC) - **Goal:** The network will provide proof of concept for incentivised store protocol +- **Deliverables**: (3) - **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization - **Estimated date of completion:** 2024Q2 - **Estimated effort:** @@ -139,6 +155,7 @@ A POC incentivisation mechanism that incorporates POC versions of the three Waku ### General Service Protocol Incentivisation (first iteration/POC) - **Goal:** The network will provide proof of concept for incentivised service protocols +- **Deliverables**: (3) - **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization - **Estimated date of completion:** 2024Q2 - **Estimated effort:** @@ -155,6 +172,7 @@ This expands store incentivisation to other protocols: ### Roadmap Towards Incentivisation on Mainnet - **Goal:** Publish a breakdown clarifying the roadmap to push incentivization to mainnet. +- **Deliverables**: (3) - **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization - **Estimated date of completion:** 2024Q3 - **Estimated effort:** @@ -169,6 +187,7 @@ The research and design necessary to come up with a roadmap to productionised in ### Capability Service Discovery - **Goal:** After this upgrade, the network will support decentralised service (e.g. store, filter) discovery on shards. +- **Deliverables**: (2), (3) - **Research tracks**: Secure Scaling, Restricted Run - **Estimated date of completion:** 2024Q2 - **Estimated effort:** @@ -186,6 +205,7 @@ See https://github.com/waku-org/research/issues/74. ### Content Service Discovery - **Goal:** After this upgrade, the network will support service discovery on content topics +- **Deliverables**: (1), (2), (3) - **Research tracks**: Secure Scaling, Restricted Run - **Estimated date of completion:** 2024Q4 - **Estimated effort:** @@ -202,6 +222,7 @@ include advertising on a separate pubsub topic, advertisements published on the ### RLN in resource-restricted clients - **Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance. +- **Deliverables**: (1), (~2), (3) - **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting - **Estimated date of completion:** 2024Q3 - **Estimated effort:** @@ -216,6 +237,7 @@ became the key focus of this effort. ### RLNv2 - **Goal:** Improved flexibility of the rate limit (from 1 msg/epoch to N msg/epoch), providing better dimensioning for bandwidth capping. +- **Deliverables**: (1), (~2), (3) - **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting - **Estimated date of completion:** 2024Q3 - **Estimated effort:** @@ -229,6 +251,7 @@ See: [https://github.com/waku-org/research/issues/22](https://github.com/waku-or ### Maturing RLN variables/parameters revision (staking, contract/chain, token) - **Goal:** A review of RLN security parameters and functionality in preparation for mainnet deployment. +- **Deliverables**: (~2), (3) - **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting - **Estimated date of completion:** 2024Q3 - **Estimated effort:** @@ -270,15 +293,22 @@ See [https://github.com/waku-org/research/issues/69](https://github.com/waku-org ### Node Bandwidth Management Mechanism +- **Deliverables**: (1), (2) + + Finish https://github.com/waku-org/pm/issues/66. ### PostgreSQL Maintenance +- **Deliverables**: (2) + Work on how to best handle PostgreSQL database growth and pruning is in progress and must completed to enable node operators to handle database growth in an easy and non-disruptive manner. ### Bindings (NodeJS, Rust, Golang) +- **Deliverables**: (1) + Provide functioning bindings over nwaku in the following environments: - NodeJS @@ -287,10 +317,14 @@ Provide functioning bindings over nwaku in the following environments: ### Bindings - Mobile +- **Deliverables**: (1) + Verify and demonstrate that the Golang and Rust bindings work on mobile environments (Android and iOS). ### Serve Limited Content Topic (Store) +- **Deliverables**: (1), (2) + Enable store service to only serve specific content topics. Note that decentralized peer discovery for content topics is scheduled at a later stage, meaning that application developers would likely need to pass or hardcode list of store node or an enrtree in their client. @@ -302,12 +336,16 @@ Ref: https://github.com/waku-org/pm/issues/64 ### User Pays Own RLN Membership +- **Deliverables**: (1) + Provide a developer-ready API for a user to generate their own credentials and insert them. Ref: (A) https://github.com/waku-org/pm/issues/102 ### Project Pays RLN Memberships for Users +- **Deliverables**: (1) + Provide a developer-ready API and software to enable developers to fund membership for users. Some concession regarding the privacy of the developer/project infra may be acceptable (e.g. project hosts a REST API to exchange private commitment). @@ -316,6 +354,8 @@ Ref: (B) https://github.com/waku-org/pm/issues/102 ### Special User Pays RLN Memberships for Other Users +- **Deliverables**: (1), (~2) + This is a continuation of [Project Pays RLN Memberships for Users](#project-pays-rln-memberships-for-users). However, requirements are stricter to preserve the privacy of the special user. @@ -326,6 +366,8 @@ Ref: (C) https://github.com/waku-org/pm/issues/102 ### RLN Credentials Security and Multiple Devices +- **Deliverables**: (1), (~2) + Define how users should handle their credentials and provide the tooling for it. Define whether RLN credentials should be shared between application or devices. May also scope the work to enable sharing credentials between devices, or done as separate milestone. @@ -334,6 +376,8 @@ Note: Support needed from RLNP2P team. ### Composing Waku Protocols to Improve Reliability +- **Deliverables**: (1), (2) + Provide recommendation in a form of a library to compose the Waku protocols to minimize message loss at a protocol level. This is both for relay and non-relay node. The aim is to provide opinionated libraries that use Waku in a generalized manner. @@ -347,11 +391,15 @@ This is part of the general SDK strategy and expected to be implemented in each ### Fault-Tolerant Usage of Web3 Providers +- **Deliverables**: (1), (~2), (3) + Review the usage of Web3 Providers and provide a fault-tolerant strategy to ensure that The Waku Network cannot be DOS'd by Web3 Provider outage. Strategies such as the software ability to fallback to alternative providers or usage of Ethereum Light Client and Portal network should be considered. ### Waku Relay and Discv5 in a Hostile, Restricted or Subpar Online Network +- **Deliverables**: (1), (3) + Proceed with a multi-client analysis of Waku Relay's performance in a somewhat online hostile or subpar network and the ability for a relay node to remain connected when said network: - Hackathon with poor internet connectivity - prevent incoming connections @@ -363,42 +411,58 @@ Note: Bluetooth, network mesh and internet outage are not in the scope of this m ### TLS-less Browser Connections +- **Deliverables**: (1), (3) + Enable node operators to serve js-waku browser clients without acquiring a domain name. Note: WebRTC Direct and webtransport support needed in nim-libp2p and js-libp2p. ### Relay in The Browser +- **Deliverables**: (3) + Understand and progress on the viability of using Waku Relay in the browser. Note: Support from js-libp2p maintainer needed. ### Waku React Native +- **Deliverables**: (1) + Provide a functioning React Native library on both Android and iOS. Likely using js-waku to enable multi-platform development (browser, iOS, Android). ### JSON RPC Deprecation +- **Deliverables**: (1) + Deprecate nwaku and go-waku JSON RPC API and replace its usage with REST API across the repos (tests). ### General Status Support +- **Deliverables**: (2) + Prioritize Status requirements, bugs or issues. ### Developer Experience Feedback +- **Deliverables**: (1) + Expected to receive and action feedback from the usage of Waku libraries. Feedback may be consolidated in a milestone. ### Operator Feature Requests +- **Deliverables**: (2) + Not a milestone per se. Review and potentially implement operator request feature requests. Requests may come from Status Community operators. ### Ad hoc Network Formation +- **Deliverables**: (1), (3) + Design and implement a protocol that uses TWN to negotiate the creation of an alternative small network that would enable lower-latency or higher bandwidth transfer. ### Future Milestones @@ -411,14 +475,20 @@ Design and implement a protocol that uses TWN to negotiate the creation of an al ### Static Sharding Integration in Status +- **Deliverables**: (2) + Complete static sharding integration in Status app and provide support for any issue encountered. ### Status Community Scaling +- **Deliverables**: (2) + See https://github.com/vacp2p/research/issues/177 ### Use of nwaku in status-go +- **Deliverables**: (~2) + Identify, plan and start the execution of replacing go-waku with nwaku in status-go. Output for 2024 is a feasibility study is done, action plan is drafted and work has started. @@ -427,6 +497,8 @@ Yet, full dependency on said libraries should not be blocking. ### Chat SDK PoC 1 +- **Deliverables**: (1) + Provide and consume JS library that enable a project to use Waku for chat purposes over TWN. Exact scope to be defined later. Likely either existing Status Chat 1:1 protocol or [Ethereum Address to Secure Channel](#ethereum-address-to-secure-channel). @@ -434,6 +506,8 @@ This library usage should be demonstrated in a web app a la wakuplay as part of ### Extract 1 Status protocol from status-go +- **Deliverables**: (1) + This may or may not be the same item as [Chat SDK PoC 1](#chat-sdk-poc-1) as the consideration to be made are different (market potential vs engineering feasibility within status-go context). @@ -442,12 +516,16 @@ future architecture in mind. ### Minimum Viable Data Synchronization +- **Deliverables**: (1) + Provide a Minimum Viable Data Synchronization protocol that developer can use to ensure the delivery of messages in their protocol. Note the [protocol](https://rfc.vac.dev/spec/2/) and a [Golang library](https://github.com/vacp2p/mvds) already exist. The aim is to review and improve the protocol and library using Status feedback, implement the protocol for the browser and provide documentation. ### Ethereum Address to Secure Channel +- **Deliverables**: (1) + Working MVP of https://rfc.vac.dev/spec/70/. ### Future Milestones @@ -460,6 +538,8 @@ Working MVP of https://rfc.vac.dev/spec/70/. ### External Wallet Authentication and Transaction for Webapp +- **Deliverables**: (1) + Enable a web app to use Waku to request wallet signature for authentication and transaction purposes. ## Vac work Required From 8849cc56886f9ecde3e75403a2f4a2af96dad1d3 Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Wed, 31 Jan 2024 10:32:48 +1100 Subject: [PATCH 12/13] Add tracking issue --- ROADMAP.md | 1 + 1 file changed, 1 insertion(+) diff --git a/ROADMAP.md b/ROADMAP.md index 9dafa48..c78bd45 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -337,6 +337,7 @@ Ref: https://github.com/waku-org/pm/issues/64 ### User Pays Own RLN Membership - **Deliverables**: (1) +- **Tracking issue**: https://github.com/waku-org/pm/issues/108 Provide a developer-ready API for a user to generate their own credentials and insert them. From 3bee35b352dca6b100131fb7dfb99b93091f7baa Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Wed, 31 Jan 2024 10:34:34 +1100 Subject: [PATCH 13/13] Mark as draft, removed estimations --- ROADMAP.md | 49 +++++-------------------------------------------- 1 file changed, 5 insertions(+), 44 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index c78bd45..8431521 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,10 +1,11 @@ -# Waku Roadmap 2024 +# Waku Roadmap 2024 (Draft) + +This live document acts as a high overview of the work to be tackled for Waku in 2024. + +**This is still a draft**. -This live document acts as a high overview of the work likely to be tackled for Waku. Actual planning and execution is done through GitHub issues and [milestones](https://github.com/waku-org/pm/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Amilestone). -MW means _man-week_: Effort time for an individual to complete the project. - ## Deliverable Waku commits to 3 deliverables (WIP): @@ -24,8 +25,6 @@ not part of the list of requirements for Status in 2024. - **Deliverables**: (3) - **Research tracks:** All - **Estimated date of completion:** 2024Q4 -- **Estimated effort:** - - research: 4 MW As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). The document itself may source from one or more academic papers published and presented throughout the year. @@ -37,9 +36,6 @@ Total length should be around 15 pages. - **Deliverables**: (3) - **Research tracks:** All - **Estimated date of completion:** 2024Q1 -- **Estimated effort:** - - research: 3 MW - - docs/eco-dev: 1 MW Review our Waku RFC strategy, review the content of the most important RFCs, improve RFC indexing and ensure that important RFCs are easily identified and accessible. @@ -84,11 +80,6 @@ Review the most important Waku RFCs: - **Deliverables**: (2), (3) - **Research tracks:** Message Reliability - **Estimated date of completion:** 2024Q2 -- **Estimated effort:** - - research: 5 MW - - nwaku - - js-waku - - docs/eco-dev: An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol: @@ -108,12 +99,6 @@ semi-centralised (trusted) service nodes to a decentralised service capability i - **Deliverables**: (2), (3) - **Research tracks:** Message Reliability - **Estimated date of completion:** 2024Q2 -- **Estimated effort:** - - research: 9 MW - - nwaku: - - js-waku: - - go-waku: - - docs/eco-dev Building on Store v3-beta, this version of Store includes basic synchronisation between nodes. This will probably include: @@ -136,11 +121,6 @@ IMO (iii) should be pursued as the preferred option, as far as possible. - **Deliverables**: (3) - **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization - **Estimated date of completion:** 2024Q2 -- **Estimated effort:** - - research: 5 weeks - - nwaku - - js-waku - - docs/eco-dev: A POC incentivisation mechanism that incorporates POC versions of the three Waku service incentivisation elements: @@ -158,11 +138,6 @@ A POC incentivisation mechanism that incorporates POC versions of the three Waku - **Deliverables**: (3) - **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization - **Estimated date of completion:** 2024Q2 -- **Estimated effort:** - - research: 5 weeks - - nwaku - - js-waku - - docs/eco-dev: This expands store incentivisation to other protocols: @@ -175,8 +150,6 @@ This expands store incentivisation to other protocols: - **Deliverables**: (3) - **Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization - **Estimated date of completion:** 2024Q3 -- **Estimated effort:** - - research: 3 weeks The research and design necessary to come up with a roadmap to productionised incentivisation, including: @@ -190,9 +163,6 @@ The research and design necessary to come up with a roadmap to productionised in - **Deliverables**: (2), (3) - **Research tracks**: Secure Scaling, Restricted Run - **Estimated date of completion:** 2024Q2 -- **Estimated effort:** - - research: 6 MW - - ... Implement a strategy for Waku nodes to discover and maintain a healthy connectivity to the set of service peers necessary for their configured applications. In the Waku Network nodes can opt to provide a range of services @@ -208,8 +178,6 @@ See https://github.com/waku-org/research/issues/74. - **Deliverables**: (1), (2), (3) - **Research tracks**: Secure Scaling, Restricted Run - **Estimated date of completion:** 2024Q4 -- **Estimated effort:** - - research 6 MW Implement a strategy for Waku nodes to discover service peers necessary for their configured applications based on the content topics configured on those service nodes. In the Waku Network nodes can opt to provide a range of services @@ -225,9 +193,6 @@ include advertising on a separate pubsub topic, advertisements published on the - **Deliverables**: (1), (~2), (3) - **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting - **Estimated date of completion:** 2024Q3 -- **Estimated effort:** - - research: 8 weeks - - js-waku... Enabling RLN proof generation and verification within light clients. Different options are set out in [https://github.com/waku-org/research/issues/45](https://github.com/waku-org/research/issues/45). @@ -240,8 +205,6 @@ became the key focus of this effort. - **Deliverables**: (1), (~2), (3) - **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting - **Estimated date of completion:** 2024Q3 -- **Estimated effort:** - - research: 4 weeks Moving from RLNv1 to RLNv2 to allow better bandwidth dimensioning in the network. This will allow a message allocation per day per registered publisher, providing better statistical guarantees for network bandwidth usage. @@ -254,8 +217,6 @@ See: [https://github.com/waku-org/research/issues/22](https://github.com/waku-or - **Deliverables**: (~2), (3) - **Research tracks:** Secure Scaling, Restricted Run, Rate Limiting - **Estimated date of completion:** 2024Q3 -- **Estimated effort:** - - research: 3 weeks Analyse RLN deployment in TWN Gen 0 and evaluate its DoS protection performance.