From 755e696099ae343c41fffde207d7c47ac421d1bc Mon Sep 17 00:00:00 2001 From: "fryorcraken.eth" Date: Thu, 25 Jan 2024 21:34:26 +1100 Subject: [PATCH] Integrate Research milestones. --- ROADMAP.md | 307 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 298 insertions(+), 9 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 5f39b2a..a146004 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,16 +1,278 @@ # Waku Roadmap This live document acts as a high overview of the work likely to be tackled for Waku. - -This is not a commitment or planning document, it is aimed to be used as a discussion starter to prioritized and plan -work. Actual planning and execution is done through GitHub issues and [milestones](https://github.com/waku-org/pm/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Amilestone). -Definition and scoping strategies for a milestone are TBD. +MW means _man-week_: Effort time for an individual to complete the project. ## Research Milestones -WIP +### The Waku Whitepaper + +**Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols +**Research tracks:** All +**Estimated date of completion:** 2024Q4 +**Estimated effort:** +- research: 4 MW + +As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7). +The document itself may source from one or more academic papers published and presented throughout the year. +Total length should be around 15 pages. + +### Waku RFC Review + +**Goal:** On completion, the set of RFCs for Waku will be simplified, reasonably up to date and accessible. +**Research tracks:** All +**Estimated date of completion:** 2024Q1 +**Estimated effort:** +- research: 3 MW +- docs/eco-dev: 1 MW + +Review our Waku RFC strategy, review the content of the most important RFCs, improve RFC indexing and ensure that +important RFCs are easily identified and accessible. + +2 outputs are expected to achieve this milestone: + +#### Review and implement RFC strategy + +**Ownership note**: this Epic is a collaboration between Waku Research (R/A) and Vac RFCs (C/I) + +Consult with stakeholders (Waku teams, Waku Community and Vac RFC team) to come up with an improved strategy for Waku RFCs to answer: + +1. Should Waku RFCs be in a separate Waku-owned repo? +2. What is the role of proof of concepts vs raw RFCs? +3. How should Waku RFCs be indexed? +4. How should Waku RFCs be presented to make important specs obvious and easy to find? +5. Should we continue with the same RFC categories (Standards, Informational and Best Current Practices)? +6. Should we continue with the same RFC tags (Waku core vs application)? +7. Should we continue with the same COSS lifecycle? Should we add a "stagnant" status a la EIPs? +8. How do we clarify the responsibility of the Vac team vs Waku team when it comes to RFCs? +9. How can external contributors contribute or propose improvements to Waku specifications? +10. Clarify intended scope of topics that can be covered in Waku RFCs, delineate categories clearly and revise the spec lifecycles. + +This item includes actioning the new proposed strategy, such as potentially moving Waku specs to a waku-org repo, +updating references to these in documentation, adding/removing tags and categories, writing up the proposed strategy +in contributor guidelines, etc. + +#### Review Waku RFCs + +**Ownership note**: this Epic is a collaboration between Waku Research (R/A) and Vac RFCs (C/I) + +Review the most important Waku RFCs: + +- review language for clarity +- ensure core specifications reflect the latest thinking around protocols and procedures +- consider marking some specifications as "stagnant" indicating that they're not being focused on +- deprecate RFCs that are no longer relevant + +### Store v3-beta - message hashes + +**Goal:** After this upgrade, the network will provide distributed and synchronised store services. +**Research tracks:** Message Reliability +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 5 MW +- nwaku +- js-waku +- docs/eco-dev: + +An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol: + +- introduces the concept of deterministic message hashes to index messages +- considers the Store as a key-value store +- allows for querying a list of keys (message hashes) from the Store +- allows for querying for the full message content (values) of a set of keys from the Store +- keeps all previous value-based filtering (e.g. content topic, timestamp) in place + +The [proposed PR](https://github.com/waku-org/waku-proto/pull/10/files) to simplify the Store protocol and use message +hashes as index/cursor, can be used as a starting point. + +### Store v3 - store synchronisation + +**Goal:** Upgrade the Store service capability in the network from a collection of local, unsynchronised, +semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation. +**Research tracks:** Message Reliability +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 9 MW +- nwaku: +- js-waku: +- go-waku: +- docs/eco-dev + +Building on Store v3-beta, this version of Store includes basic synchronisation between nodes. +This will probably include: + +- a protocol/heuristic to resume store services after an offline period +- a protocol/heuristic to periodically compare local key-value store with other nodes and find missing keys +- a protocol/heuristic to periodically download the messages (values) for missing keys from other store nodes + +Note that this can either be + +- (i) designing a new synchronisation protocol built into the Store protocol +- (ii) adapting existing synchronisation building blocks (e.g. a [Prolly Tree library](https://docs.canvas.xyz/blog/2023-05-04-merklizing-the-key-value-store.html)) into Store, or +- (iii) simply swapping the key-value store itself to a synchronised backend, such as [GossipLog](https://github.com/canvasxyz/canvas/tree/c1c208032b56d6b474996cb7da7bda5c55b69463/packages/gossiplog) + +IMO (iii) should be pursued as the preferred option, as far as possible. + +### Store Incentivisation (first iteration/POC) + +**Goal:** The network will provide proof of concept for incentivised store protocol +**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 5 weeks +- nwaku +- js-waku +- docs/eco-dev: + +A POC incentivisation mechanism that incorporates POC versions of the three Waku service incentivisation elements: + +- a price offer/negotiation mechanism +- a proof of payment system +- a local reputation mechanism to distinguish between "good" and "bad" store nodes + +> Note: Milestones in the incentivization domain may be significantly disrupted as the overall direction here is still being clarified re topics +> such as tokenomics, decisions about monetary vs reputational incentivisation, etc. Similarly, distributed reputation +> may soon form a very large part of this effort, but is not listed here as it's unclear where/if it fits the roadmap. + +### General Service Protocol Incentivisation (first iteration/POC) + +**Goal:** The network will provide proof of concept for incentivised service protocols +**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 5 weeks +- nwaku +- js-waku +- docs/eco-dev: + +This expands store incentivisation to other protocols: + +- consider the most reasonable incentivisation vectors for non-store service protocols, if they need incentivisation at all (filter, lightpush, peer-exchange) +- apply the POC incentivisation elements developed for Store to other service protocols where applicable, delineating alternative strategies where not. + +### Roadmap Towards Incentivisation on Mainnet + +**Goal:** Publish a breakdown clarifying the roadmap to push incentivization to mainnet. +**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 3 weeks + +The research and design necessary to come up with a roadmap to productionised incentivisation, including: + +1. evaluating the POC incentivisation mechanisms for fairness, security, exploitability and changing tactics where necessary +2. maturing our tokenomics, etc +3. establishing the (possible) role of a global reputation mechanism + +### Capability Service Discovery + +**Goal:** After this upgrade, the network will support decentralised service (e.g. store, filter) discovery on shards. +**Research tracks**: Secure Scaling, Restricted Run +**Estimated date of completion:** 2024Q2 +**Estimated effort:** +- research: 6 MW +- ... + +Implement a strategy for Waku nodes to discover and maintain a healthy connectivity to the set of service peers +necessary for their configured applications. In the Waku Network nodes can opt to provide a range of services +(Store, Filter, etc.). Nodes can provide these services for one or more of the 8 pubsub topics comprising the Waku +Network. Any peer in the network trying to make use of these services should be able to easily find a service node +matching its desired (shard, service) capability vector. Various service discovery approaches is possible here. + +See https://github.com/waku-org/research/issues/74. + +### Content Service Discovery + +**Goal:** After this upgrade, the network will support service discovery on content topics +**Research tracks**: Secure Scaling, Restricted Run +**Estimated date of completion:** 2024Q4 + +**Estimated effort:** +- research 6 MW + +Implement a strategy for Waku nodes to discover service peers necessary for their configured applications based on the +content topics configured on those service nodes. In the Waku Network nodes can opt to provide a range of services +(Store, Filter, etc.). Nodes can provide these services for any set of content topic(s), allowing application-specific +service operators and nodes. Any peer in the network trying to make use of these services should be able to easily find +a service node matching its desired set of content topics. Note that the cardinality of all possible content topics +(infinite) may be too high for integration into existing DHTs/shared discovery methods. Alternative approaches could +include advertising on a separate pubsub topic, advertisements published on the configured content topics, etc. + +## Milestone: TWN - Rate-limiting Upgrade (RU) + +**Goal:** After this upgrade, the network will show maturity in how it limits rate, with better dimensioning for +bandwidth capping, resource-restricted options and a clear evaluation of what is needed to improve DoS protection. +**[Research tracks:](https://www.notion.so/Waku-Research-ac7941c0ece84dc6b2948ec02b3cb695?pvs=21)** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 30/09/2024 + +### RLN in resource-restricted clients + +**Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance. +**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 8 weeks +- js-waku... + +Enabling RLN proof generation and verification within light clients. +Different options are set out in [https://github.com/waku-org/research/issues/45](https://github.com/waku-org/research/issues/45). +Most recently, the promising option of deploying the entire RLN membership tree to a L2 as opposed to a L1 chain, +became the key focus of this effort. + +### RLNv2 + +**Goal:** Improved flexibility of the rate limit (from 1 msg/epoch to N msg/epoch), providing better dimensioning for bandwidth capping. +**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 4 weeks + +Moving from RLNv1 to RLNv2 to allow better bandwidth dimensioning in the network. This will allow a message allocation +per day per registered publisher, providing better statistical guarantees for network bandwidth usage. + +See: [https://github.com/waku-org/research/issues/22](https://github.com/waku-org/research/issues/22) for more + +### Maturing RLN variables/parameters revision (staking, contract/chain, token) + +**Goal:** A review of RLN security parameters and functionality in preparation for mainnet deployment. +**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting +**Estimated date of completion:** 2024Q3 +**Estimated effort:** +- research: 3 weeks + +Analyse RLN deployment in TWN Gen 0 and evaluate its DoS protection performance. + +- should staking be introduced, especially to improve resilience against adversarial membership registrations? +- should slashing be introduced or does the existing gossipsub scoring method provide enough protection? +- which chain or L2 should we target for memberships? +- if staking is introduced, which token should be used? +- do we need a combination of msg/sec and msg allocation/day rate limiting? + +### Unscheduled Milestones + +The following items are not yet scheduled, but may be prioritised in the first few months of 2024: + +- publish integrated peer and connection management strategy for all Waku protocols +- implement a distributed/global reputation mechanism +- pluggable validator strategy (see [https://github.com/waku-org/nwaku/issues/1700](https://github.com/waku-org/nwaku/issues/1700)) +- general Waku protocol security review and audit + +### Future Milestoness + +#### Decentralised bootstrapping + +Improve DNS-based discovery decentralisation by: + +1. including non-Waku nodes in the default bootstrap list/domain published by Waku +2. allowing non-Waku parties to easily create and publish their own lists, likely in the form of a discv5 DHT crawler that creates a DNS-discovery compatible Merkle tree of discovered nodes, signs and publishes these to a domain +3. using the DNS-discovery link tree functionality to link multiple published lists +4. allowing (and creating?) offline bootstrap lists maintained locally for each node + +See [https://github.com/waku-org/research/issues/69](https://github.com/waku-org/research/issues/69) for more + +### RLNv3 ## Engineering Milestones @@ -31,14 +293,18 @@ Provide functioning bindings over nwaku in the following environments: - Rust (swap go-waku with nwaku) - Golang +### Bindings - Mobile + +Verify and demonstrate that the Golang and Rust bindings work on mobile environments (Android and iOS). + ### Serve Limited Content Topic (Store) Enable store service to only serve specific content topics. Note that decentralized peer discovery for content topics is scheduled at a later stage, meaning that application developers would likely need to pass or hardcode list of store node or an enrtree in their client. -Risk: Need to cater for scenarios where a node connect to a store that does not support their content topic. -TODO: Should a small change to metadata protocol be planned? Should this be a research item instead? +Need to cater for scenarios where a node connect to a store that does not support their content topic, such as a +possible update of the metadata protocol. Ref: https://github.com/waku-org/pm/issues/64 @@ -143,7 +409,7 @@ Requests may come from Status Community operators. Design and implement a protocol that uses TWN to negotiate the creation of an alternative small network that would enable lower-latency or higher bandwidth transfer. -### Unscheduled +### Future Milestones - Serve only Content Topic Specific messages (filter, light push) - Merge js-waku and NodeJS-nwaku APIs to enable usage of single SDK for JavaScript on all environments. @@ -192,7 +458,7 @@ The aim is to review and improve the protocol and library using Status feedback, Working MVP of https://rfc.vac.dev/spec/70/. -### Unscheduled – 2025 Priority +### Future Milestones - Private group chat SDK - Wakuplay usage to dogfood private group chat @@ -203,3 +469,26 @@ Working MVP of https://rfc.vac.dev/spec/70/. ### External Wallet Authentication and Transaction for Webapp Enable a web app to use Waku to request wallet signature for authentication and transaction purposes. + +## Vac work Required + +**Ownership note**: these items are owned by Vac (R/A) and supported by Waku Research (C/I) + +1. Continue DST simulations: simulation of Waku network of 10K users, RLN simulations, etc. (DST) +2. Continue QA work: improve test coverage from a RFC and Status usage angle, including interoperability test run + between nwaku and go-waku, and once done, js-waku and nwaku. Considerations should be made to also include status-go (DST). +3. Simulate or model status-go behaviour based on the result of Waku large scale simulations. (DST) +4. Secure RLN credentials: for example in [CryptKeeper](https://github.com/CryptKeeperZK) (RLNP2P) +5. Support in improving RLN usage on resource-restricted devices (RLNP2P) +6. Secure way of delegating acquisition of memberships as described in [RLN Design Patterns](https://github.com/waku-org/pm/issues/102) (RLNP2P) +7. Fee structure for RLN memberships: come up with a comprehensive strategy for RLN memberships, perhaps following + the [NFT-like approach](https://forum.vac.dev/t/ens-like-nfts-for-rln/243). Strategy should consider cost structures + like Harberger tax, dev/node compensation and project sustainability, security, etc (RLNP2P, Tokenomics, Smart Contract) +8. RLN contract audited, reviewed and deployed to mainnet (RLNP2P, Smart Contract) +9. Investigate feasibility/risks for RLN on L2, recommend a specific L2 if any (RLNP2P, Tokenomics, Smart Contract) +10. Risk/attack analysis for RLN. How do we recover after an attack? (RLNP2P, Tokenomics) +11. Deliver a cryptographic protocol to enable setting up a secure channel for 1:1 and private group chat purposes from + an Ethereum address (ACZ). +12. Support in terms of cryptographic security, especially in regards of extending the Waku noise protocol if such a + need arises from Waku adoption (ACZ). +