logos-messaging/pm
1
0
Fork 0
mirror of https://github.com/logos-messaging/pm.git synced 2026-01-02 14:13:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Incorporate feedback and focus on two features for now

Browse Source 
# Conflicts:
#	requirements/codex.md
This commit is contained in:
fryorcraken 2025-07-24 12:27:16 +10:00
parent 82d33b63e1
commit b703f4dbf7
No known key found for this signature in database
GPG Key ID: A82ED75A8DFC50A4
1 changed files with 54 additions and 61 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

115
requirements/codex.md
View File

@ -1,112 +1,105 @@
# Waku's Requirements on Codex
## Publish Large Messages - Uploader is online
## Message Archival
To be used for messages archival in Chat SDK, Qaku, opchan, etc.
It assumes that a special user (admin) regularly bundles messages and pushes them to an external system.
It then pushes the CID (or any other reference to retrieve the bundle) over Waku.
It assumes that a special user (admin, referred to as "original uploader") regularly bundles messages and pushes them to an external system.
It then pushes the reference to the bundle over Waku.
New users retrieve and listen to new messages using Waku.
Thanks to SDS, they learn whether they miss messages, and if so, can proceed with retrieval from the latest bundle.
New users retrieve and listen to new messages using Waku upon start up.
Then, they may retrieve bundles, likely because the know they are missing message via SDS.
(clearly, spec is needed).
The original uploader is the one to determine durability, aka, relevance of data over time.
It is application specific (eg until a Q&A is completed), and not related to users having downloaded the data.
Which means it's a scenario where:
- Time from upload to retrieval is **not** critical (latest messages are available on Waku)
- Several users can seed and download the bundle.
This is very similar to BitTorrent integration in Status. I need to find specs to be more explicit about difference.
Some notes on BitTorrent integration in Status (AFAIK, asking @osmaczko for help):
1. Known issue is that the bundle is very large, and hence consumes a lot of bandwidth. I don't know if the bundle is "updated" or overridden.
On Waku app side, we need to be open to have one large bundle vs a series of manageable bundle. The latter offers more flexibility such as attaching bloom filters,
for selective download.
2. The bundle download is indiscriminate, meaning every user will download it at some point, with SDS, we can do something smarter
Also note (more of a personal opinion), usage of BitTorrent/webtorrent could be an acceptable starting point.
### Functionality
1. Ability to transfer a message of 1MB or more between two or more nodes.
2. Message's CID is less than 100kB.
1. Ability to transfer a bundle of 1MB or more between two or more nodes.
2. Reference to bundle is 50kB or less.
### Usability
1. Developer can implement upload feature with 10 lines of code or less.
2. No configuration is necessary.
1. Developer can implement feature with 10 lines of code or less.
### Reliability
1. Download operation can be resumed.
2. Upload operation can be resumed.
3. Uploader can be expected to be online when user are downloading.
3. As long as original developer is online, bundle should be retrievable.
4. As long as N out of M users are online, bundle should be retrievable.
### Performance
None
1. Time between bundle uploaded, and retrieved by users can be in the span of minutes and hours (we assumes messages are available in Waku store for several hours).
2. The burden of re-upload is shared by users.
### Supportability
1. Library for Browser applications.
2. Library for Nim desktop applications.
3. Library for Nim mobile applications.
4. Most users may be behind NAT routers and other domestic network setup.
**+ (Privacy, Anonymity, Censorship-Resistance, Deployments)**
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)
1. The unavailability of a static host (IP, DNS) does not prevent a user to upload or download (censorship-resistance).
2. TODO (privacy)
2. A participant cannot determine original uploader's PII (anonymity).
## Publish Large Messages - Uploader is offline
## Large File Transfer
To be used for
To be used when 2 users or more, are transferring a large payload. This may be a large image or video in a private chat.
Or it could be a llm prompt that returns a large image or video.
- large messages transfers (such as images, videos, audio) in Chat SDK, Opchan, etc.
- Enhancement of message archival (uploader does not need to be online for messages to be retrieved).
Due to the broadcast nature of Waku, it would hog too much bandwidth if every large file sent between users where sent over Waku.
Builds on [Publish Large Messages - Uploader is online](#publish-large-messages---uploader-is-online)
In terms of durability, it can be assumed that once all participants have downloaded the payload, it does not need to be retrievable anymore.
It should also be assumed that the users may not be online at the same time (mobile).
There is more expectation on timeliness of retrievability, as one would want to be able to download seconds after the upload happened.
### Functionality
1. After upload, message is retrievable without sender being online.
2. Best effort in terms of message retention; expectations on restrictions are documented.
1. Ability to transfer a payload of 1MB or more between two or more peers.
2. Reference to payload is less than 50kB.
### Usability
1. Receiver can download the large message, even if sender is offline, as long as they get the CID out-of-band.
1. Developer can implement feature with 10 lines of code or less.
### Reliability
1. Uploader may be offline when receiver is retrieving the large message.
1. Download operation can be resumed.
2. Upload operation can be resumed.
3. Payload should be retrievable even if original uploader goes offline.
4. Once all recipients have downloaded the payload, there is no more durability expectations.
### Performance
None
1. Payload download should start within seconds of the upload start.
### Supportability
1. Library for Browser applications.
2. Library for Nim applications.
2. Library for Nim desktop applications.
3. Library for Nim mobile applications.
4. Most users may be behind NAT routers and other domestic network setup.
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)**
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)
TODO (privacy)
## Publish Large Messages - Retention is guaranteed
In this scenario, the uploader wants to ensure the data is persisted and is willing to pay for it.
This may be a Qaku Q&A admin, a opchan cell owner or Status Communities owner.
Builds on previous requirements.
### Functionality
1. Uploader may pay for large message storage to have guaranteed retention.
### Usability
1. Receiver can download the large message, even if sender is offline, as long as they get the CID out-of-band.
2. Receiver does not need to pay to retrieve the large message.
### Reliability
1. Uploader may be offline when receiver is retrieving the large message.
2. Uploader is guaranteed a period of retention for a given price.
### Performance
See previous requirements.
### Supportability
See previous requirements.
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)**
See previous requirements.
1. The unavailability of a static host (IP, DNS) does not prevent a user to upload or download (censorship-resistance).
2. An external observer cannot tie the PIIs of the uploader and downloaders of one payload;
it is assumed that the reference to the payload (eg, CID) is not leaked outside the participants.