logos-messaging/pm
1
0
Fork 0
mirror of https://github.com/logos-messaging/pm.git synced 2026-01-03 14:43:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

Integrate Research milestones.

Browse Source
This commit is contained in:
fryorcraken.eth 2024-01-25 21:34:26 +11:00
parent 6c916e4c77
commit 755e696099
No known key found for this signature in database
GPG Key ID: A82ED75A8DFC50A4
1 changed files with 298 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

307
ROADMAP.md
View File

@ -1,16 +1,278 @@
# Waku Roadmap
This live document acts as a high overview of the work likely to be tackled for Waku.
This is not a commitment or planning document, it is aimed to be used as a discussion starter to prioritized and plan
work.
Actual planning and execution is done through GitHub issues and [milestones](https://github.com/waku-org/pm/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Amilestone).
Definition and scoping strategies for a milestone are TBD.
MW means _man-week_: Effort time for an individual to complete the project.
## Research Milestones
WIP
### The Waku Whitepaper
**Goal:** On completion, Waku will have an academically rigorous whitepaper explaining the what, why and how of Waku protocols
**Research tracks:** All
**Estimated date of completion:** 2024Q4
**Estimated effort:**
- research: 4 MW
As set out in [https://github.com/waku-org/research/issues/7](https://github.com/waku-org/research/issues/7).
The document itself may source from one or more academic papers published and presented throughout the year.
Total length should be around 15 pages.
### Waku RFC Review
**Goal:** On completion, the set of RFCs for Waku will be simplified, reasonably up to date and accessible.
**Research tracks:** All
**Estimated date of completion:** 2024Q1
**Estimated effort:**
- research: 3 MW
- docs/eco-dev: 1 MW
Review our Waku RFC strategy, review the content of the most important RFCs, improve RFC indexing and ensure that
important RFCs are easily identified and accessible.
2 outputs are expected to achieve this milestone:
#### Review and implement RFC strategy
**Ownership note**: this Epic is a collaboration between Waku Research (R/A) and Vac RFCs (C/I)
Consult with stakeholders (Waku teams, Waku Community and Vac RFC team) to come up with an improved strategy for Waku RFCs to answer:
1. Should Waku RFCs be in a separate Waku-owned repo?
2. What is the role of proof of concepts vs raw RFCs?
3. How should Waku RFCs be indexed?
4. How should Waku RFCs be presented to make important specs obvious and easy to find?
5. Should we continue with the same RFC categories (Standards, Informational and Best Current Practices)?
6. Should we continue with the same RFC tags (Waku core vs application)?
7. Should we continue with the same COSS lifecycle? Should we add a "stagnant" status a la EIPs?
8. How do we clarify the responsibility of the Vac team vs Waku team when it comes to RFCs?
9. How can external contributors contribute or propose improvements to Waku specifications?
10. Clarify intended scope of topics that can be covered in Waku RFCs, delineate categories clearly and revise the spec lifecycles.
This item includes actioning the new proposed strategy, such as potentially moving Waku specs to a waku-org repo,
updating references to these in documentation, adding/removing tags and categories, writing up the proposed strategy
in contributor guidelines, etc.
#### Review Waku RFCs
**Ownership note**: this Epic is a collaboration between Waku Research (R/A) and Vac RFCs (C/I)
Review the most important Waku RFCs:
- review language for clarity
- ensure core specifications reflect the latest thinking around protocols and procedures
- consider marking some specifications as "stagnant" indicating that they're not being focused on
- deprecate RFCs that are no longer relevant
### Store v3-beta - message hashes
**Goal:** After this upgrade, the network will provide distributed and synchronised store services.
**Research tracks:** Message Reliability
**Estimated date of completion:** 2024Q2
**Estimated effort:**
- research: 5 MW
- nwaku
- js-waku
- docs/eco-dev:
An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol:
- introduces the concept of deterministic message hashes to index messages
- considers the Store as a key-value store
- allows for querying a list of keys (message hashes) from the Store
- allows for querying for the full message content (values) of a set of keys from the Store
- keeps all previous value-based filtering (e.g. content topic, timestamp) in place
The [proposed PR](https://github.com/waku-org/waku-proto/pull/10/files) to simplify the Store protocol and use message
hashes as index/cursor, can be used as a starting point.
### Store v3 - store synchronisation
**Goal:** Upgrade the Store service capability in the network from a collection of local, unsynchronised,
semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation.
**Research tracks:** Message Reliability
**Estimated date of completion:** 2024Q2
**Estimated effort:**
- research: 9 MW
- nwaku:
- js-waku:
- go-waku:
- docs/eco-dev
Building on Store v3-beta, this version of Store includes basic synchronisation between nodes.
This will probably include:
- a protocol/heuristic to resume store services after an offline period
- a protocol/heuristic to periodically compare local key-value store with other nodes and find missing keys
- a protocol/heuristic to periodically download the messages (values) for missing keys from other store nodes
Note that this can either be
- (i) designing a new synchronisation protocol built into the Store protocol
- (ii) adapting existing synchronisation building blocks (e.g. a [Prolly Tree library](https://docs.canvas.xyz/blog/2023-05-04-merklizing-the-key-value-store.html)) into Store, or
- (iii) simply swapping the key-value store itself to a synchronised backend, such as [GossipLog](https://github.com/canvasxyz/canvas/tree/c1c208032b56d6b474996cb7da7bda5c55b69463/packages/gossiplog)
IMO (iii) should be pursued as the preferred option, as far as possible.
### Store Incentivisation (first iteration/POC)
**Goal:** The network will provide proof of concept for incentivised store protocol
**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization
**Estimated date of completion:** 2024Q2
**Estimated effort:**
- research: 5 weeks
- nwaku
- js-waku
- docs/eco-dev:
A POC incentivisation mechanism that incorporates POC versions of the three Waku service incentivisation elements:
- a price offer/negotiation mechanism
- a proof of payment system
- a local reputation mechanism to distinguish between "good" and "bad" store nodes
> Note: Milestones in the incentivization domain may be significantly disrupted as the overall direction here is still being clarified re topics
> such as tokenomics, decisions about monetary vs reputational incentivisation, etc. Similarly, distributed reputation
> may soon form a very large part of this effort, but is not listed here as it's unclear where/if it fits the roadmap.
### General Service Protocol Incentivisation (first iteration/POC)
**Goal:** The network will provide proof of concept for incentivised service protocols
**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization
**Estimated date of completion:** 2024Q2
**Estimated effort:**
- research: 5 weeks
- nwaku
- js-waku
- docs/eco-dev:
This expands store incentivisation to other protocols:
- consider the most reasonable incentivisation vectors for non-store service protocols, if they need incentivisation at all (filter, lightpush, peer-exchange)
- apply the POC incentivisation elements developed for Store to other service protocols where applicable, delineating alternative strategies where not.
### Roadmap Towards Incentivisation on Mainnet
**Goal:** Publish a breakdown clarifying the roadmap to push incentivization to mainnet.
**Research tracks:** Secure Scaling, Restricted Run, Protocol Incentivization
**Estimated date of completion:** 2024Q3
**Estimated effort:**
- research: 3 weeks
The research and design necessary to come up with a roadmap to productionised incentivisation, including:
1. evaluating the POC incentivisation mechanisms for fairness, security, exploitability and changing tactics where necessary
2. maturing our tokenomics, etc
3. establishing the (possible) role of a global reputation mechanism
### Capability Service Discovery
**Goal:** After this upgrade, the network will support decentralised service (e.g. store, filter) discovery on shards.
**Research tracks**: Secure Scaling, Restricted Run
**Estimated date of completion:** 2024Q2
**Estimated effort:**
- research: 6 MW
- ...
Implement a strategy for Waku nodes to discover and maintain a healthy connectivity to the set of service peers
necessary for their configured applications. In the Waku Network nodes can opt to provide a range of services
(Store, Filter, etc.). Nodes can provide these services for one or more of the 8 pubsub topics comprising the Waku
Network. Any peer in the network trying to make use of these services should be able to easily find a service node
matching its desired (shard, service) capability vector. Various service discovery approaches is possible here.
See https://github.com/waku-org/research/issues/74.
### Content Service Discovery
**Goal:** After this upgrade, the network will support service discovery on content topics
**Research tracks**: Secure Scaling, Restricted Run
**Estimated date of completion:** 2024Q4
**Estimated effort:**
- research 6 MW
Implement a strategy for Waku nodes to discover service peers necessary for their configured applications based on the
content topics configured on those service nodes. In the Waku Network nodes can opt to provide a range of services
(Store, Filter, etc.). Nodes can provide these services for any set of content topic(s), allowing application-specific
service operators and nodes. Any peer in the network trying to make use of these services should be able to easily find
a service node matching its desired set of content topics. Note that the cardinality of all possible content topics
(infinite) may be too high for integration into existing DHTs/shared discovery methods. Alternative approaches could
include advertising on a separate pubsub topic, advertisements published on the configured content topics, etc.
## Milestone: TWN - Rate-limiting Upgrade (RU)
**Goal:** After this upgrade, the network will show maturity in how it limits rate, with better dimensioning for
bandwidth capping, resource-restricted options and a clear evaluation of what is needed to improve DoS protection.
**[Research tracks:](https://www.notion.so/Waku-Research-ac7941c0ece84dc6b2948ec02b3cb695?pvs=21)** Secure Scaling, Restricted Run, Rate Limiting
**Estimated date of completion:** 30/09/2024
### RLN in resource-restricted clients
**Goal:** Using RLN in resource-restricted provide fair user experience in terms of initial setup and performance.
**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting
**Estimated date of completion:** 2024Q3
**Estimated effort:**
- research: 8 weeks
- js-waku...
Enabling RLN proof generation and verification within light clients.
Different options are set out in [https://github.com/waku-org/research/issues/45](https://github.com/waku-org/research/issues/45).
Most recently, the promising option of deploying the entire RLN membership tree to a L2 as opposed to a L1 chain,
became the key focus of this effort.
### RLNv2
**Goal:** Improved flexibility of the rate limit (from 1 msg/epoch to N msg/epoch), providing better dimensioning for bandwidth capping.
**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting
**Estimated date of completion:** 2024Q3
**Estimated effort:**
- research: 4 weeks
Moving from RLNv1 to RLNv2 to allow better bandwidth dimensioning in the network. This will allow a message allocation
per day per registered publisher, providing better statistical guarantees for network bandwidth usage.
See: [https://github.com/waku-org/research/issues/22](https://github.com/waku-org/research/issues/22) for more
### Maturing RLN variables/parameters revision (staking, contract/chain, token)
**Goal:** A review of RLN security parameters and functionality in preparation for mainnet deployment.
**Research tracks:** Secure Scaling, Restricted Run, Rate Limiting
**Estimated date of completion:** 2024Q3
**Estimated effort:**
- research: 3 weeks
Analyse RLN deployment in TWN Gen 0 and evaluate its DoS protection performance.
- should staking be introduced, especially to improve resilience against adversarial membership registrations?
- should slashing be introduced or does the existing gossipsub scoring method provide enough protection?
- which chain or L2 should we target for memberships?
- if staking is introduced, which token should be used?
- do we need a combination of msg/sec and msg allocation/day rate limiting?
### Unscheduled Milestones
The following items are not yet scheduled, but may be prioritised in the first few months of 2024:
- publish integrated peer and connection management strategy for all Waku protocols
- implement a distributed/global reputation mechanism
- pluggable validator strategy (see [https://github.com/waku-org/nwaku/issues/1700](https://github.com/waku-org/nwaku/issues/1700))
- general Waku protocol security review and audit
### Future Milestoness
#### Decentralised bootstrapping
Improve DNS-based discovery decentralisation by:
1. including non-Waku nodes in the default bootstrap list/domain published by Waku
2. allowing non-Waku parties to easily create and publish their own lists, likely in the form of a discv5 DHT crawler that creates a DNS-discovery compatible Merkle tree of discovered nodes, signs and publishes these to a domain
3. using the DNS-discovery link tree functionality to link multiple published lists
4. allowing (and creating?) offline bootstrap lists maintained locally for each node
See [https://github.com/waku-org/research/issues/69](https://github.com/waku-org/research/issues/69) for more
### RLNv3
## Engineering Milestones
@ -31,14 +293,18 @@ Provide functioning bindings over nwaku in the following environments:
- Rust (swap go-waku with nwaku)
- Golang
### Bindings - Mobile
Verify and demonstrate that the Golang and Rust bindings work on mobile environments (Android and iOS).
### Serve Limited Content Topic (Store)
Enable store service to only serve specific content topics.
Note that decentralized peer discovery for content topics is scheduled at a later stage, meaning that application
developers would likely need to pass or hardcode list of store node or an enrtree in their client.
Risk: Need to cater for scenarios where a node connect to a store that does not support their content topic.
TODO: Should a small change to metadata protocol be planned? Should this be a research item instead?
Need to cater for scenarios where a node connect to a store that does not support their content topic, such as a
possible update of the metadata protocol.
Ref: https://github.com/waku-org/pm/issues/64
@ -143,7 +409,7 @@ Requests may come from Status Community operators.
Design and implement a protocol that uses TWN to negotiate the creation of an alternative small network that would enable lower-latency or higher bandwidth transfer.
### Unscheduled
### Future Milestones
- Serve only Content Topic Specific messages (filter, light push)
- Merge js-waku and NodeJS-nwaku APIs to enable usage of single SDK for JavaScript on all environments.
@ -192,7 +458,7 @@ The aim is to review and improve the protocol and library using Status feedback,
Working MVP of https://rfc.vac.dev/spec/70/.
### Unscheduled – 2025 Priority
### Future Milestones
- Private group chat SDK
- Wakuplay usage to dogfood private group chat
@ -203,3 +469,26 @@ Working MVP of https://rfc.vac.dev/spec/70/.
### External Wallet Authentication and Transaction for Webapp
Enable a web app to use Waku to request wallet signature for authentication and transaction purposes.
## Vac work Required
**Ownership note**: these items are owned by Vac (R/A) and supported by Waku Research (C/I)
1. Continue DST simulations: simulation of Waku network of 10K users, RLN simulations, etc. (DST)
2. Continue QA work: improve test coverage from a RFC and Status usage angle, including interoperability test run
between nwaku and go-waku, and once done, js-waku and nwaku. Considerations should be made to also include status-go (DST).
3. Simulate or model status-go behaviour based on the result of Waku large scale simulations. (DST)
4. Secure RLN credentials: for example in [CryptKeeper](https://github.com/CryptKeeperZK) (RLNP2P)
5. Support in improving RLN usage on resource-restricted devices (RLNP2P)
6. Secure way of delegating acquisition of memberships as described in [RLN Design Patterns](https://github.com/waku-org/pm/issues/102) (RLNP2P)
7. Fee structure for RLN memberships: come up with a comprehensive strategy for RLN memberships, perhaps following
the [NFT-like approach](https://forum.vac.dev/t/ens-like-nfts-for-rln/243). Strategy should consider cost structures
like Harberger tax, dev/node compensation and project sustainability, security, etc (RLNP2P, Tokenomics, Smart Contract)
8. RLN contract audited, reviewed and deployed to mainnet (RLNP2P, Smart Contract)
9. Investigate feasibility/risks for RLN on L2, recommend a specific L2 if any (RLNP2P, Tokenomics, Smart Contract)
10. Risk/attack analysis for RLN. How do we recover after an attack? (RLNP2P, Tokenomics)
11. Deliver a cryptographic protocol to enable setting up a secure channel for 1:1 and private group chat purposes from
an Ethereum address (ACZ).
12. Support in terms of cryptographic security, especially in regards of extending the Waku noise protocol if such a
need arises from Waku adoption (ACZ).