Autogenerate sidebar (#166)

Co-authored-by: LordGhostX <47832826+LordGhostX@users.noreply.github.com>

docs/research/benchmarks/_category_.json

@@ -0,0 +1 @@
+{ "label": "Benchmarks", "collapsed": false }

docs/research/index.md

@@ -0,0 +1,9 @@
+---
+title: Research
+hide_table_of_contents: true
+sidebar_position: 1
+---
+
+**Research and Studies**: Protocol simulations and theoretical analysis to support the design of Waku protocols. The protocol definitions are on the [Vac RFC](https://rfc.vac.dev/) website.
+
+**Benchmarks**: Results of implementations and engineering-related benchmarks for Waku clients.

docs/research/research-and-studies/_category_.json

@@ -0,0 +1 @@
+{ "label": "Research and Studies", "collapsed": false }

docs/research/research-and-studies/capped-bandwidth.md

@@ -0,0 +1,64 @@
+---
+title: Capped Bandwidth in Waku
+---
+
+This issue explains i) why The Waku Network requires a capped bandwidth per shard and ii) how to solve it by rate limiting with RLN by daily requests (instead of every x seconds), which would require RLN v2, or some modifications in the current circuits to work. It also explains why the current rate limiting RLN approach (limit 1 message every x seconds) is not practical to solve this problem.
+
+## Problem
+
+First of all, lets begin with the terminology. We have talked in the past about "predictable" bandwidth, but a better name would be "capped" bandwidth. This is because it is totally fine that the waku traffic is not predictable, as long as its capped. And it has to be capped because otherwise no one will be able to run a node.
+
+Since we aim that everyone is able to run a full waku node (at least subscribed to a single shard) its of paramount importance that the bandwidth requirements (up/down) are i) reasonable to run with a residential internet connection in every country and ii) limited to an upper value, aka capped. If the required bandwidth to stay up to date with a topic is higher than what the node has available, then it will start losing messages and won't be able to stay up to date with the topic messages. And not to mention the problems this will cause to other services and applications being used by the user.
+
+The main problem is that one can't just chose the bandwidth it allocates to `relay`. One could set the maximum bandwidth willing to allocate to `store` but this is not how `relay` works. The required bandwidth is not set by the node, but by the network. If a pubsub topic `a` has a traffic of 50 Mbps (which is the sum of all messages being sent multiplied by its size, times the D_out degree), then if a node wants to stay up to date in that topic, and relay traffic in it, then it will require 50 Mbps. There is no thing such as "partially contribute" to the topic (with eg 25Mbps) because then you will be losing messages, becoming an unreliable peer. The network sets the pace.
+
+So waku needs an upper boundary on the in/out bandwidth (mbps) it consumes. Just like apps have requirements on cpu and memory, we should set a requirement on bandwidth, and then guarantee that if you have that bandwidth, you will be able to run a node without any problem. And this is the tricky part.
+
+## Current approach
+
+With the recent productisation effort of RLN, we have part of the problem solved, but not entirely. RLN offers an improvement, since now have a pseudo-identity (RLN membership) that can be used to rate limit users, enforcing a limit on how often it can send a message (eg 1 message every 10 seconds). We assume of course, that getting said RLN membership requires to pay something, or put something at stake, so that it can't be sibyl attacked.
+
+Rate limiting with RLN so that each entity just sends 1 message every x seconds indeed solves the spam problem but it doesn't per se cap the traffic. In order to cap the traffic, we would first need to cap the amount of memberships we allow. Lets see an example: 
+- We limit to 10.000 RLN memberships
+- Each ones is rate limited to send 1 message/10 seconds
+- Message size of 50 kBytes
+
+Having this, the worst case bandwidth that we can theoretically have, would be if all of the memberships publish messages at the same time, with the maximum size, continuously. That is `10.000 messages/sec * 50 kBytes = 500 MBytes/second`. This would be a burst every 10 seconds, but enough to leave out the majority of the nodes. Of course this assumption is not realistic as most likely not everyone will continuously send messages at the same time and the size will vary. But in theory this could happen.
+
+A naive (and not practical) way of fixing this, would be to design the network for this worst case. So if we want to cap the maximum bandwidth to 5 MBytes/s then we would have different options on the maximum i) amount of RLN memberships and ii) maximum message size:
+- `1.000` RLN memberships, `5` kBytes message size: `1000 * 5 = 5 MBytes/s`
+- `10.000` RLN memberships, `500` Bytes message size: `10000 * 0.5 = 5 MBytes/s`
+
+In both cases we cap the traffic, however, if we design The Waku Network like this, it will be massively underutilized. As an alternative, the approach we should follow is to rely on statistics, and assume that i) not everyone will be using the network at the same time and ii) message size will vary. So while its impossible to guarantee any capped bandwidth, we should be able to guarantee that with 95 or 99% confidence the bandwidth will stay around a given value, with a maximum variance.
+
+The current RLN approach of rate limiting 1 message every x seconds is not very practical. The current RLN limitations are enforced on 1 message every x time (called `epoch`). So we currently can allow 1 msg per second or 1 msg per 10 seconds by just modifying the `epoch` size. But this has some drawbacks. Unfortunately, neither of the options are viable for waku:
+1. A small `epoch` size (eg `1 seconds`) would allow a membership to publish `24*3600/1=86400` messages a day, which would be too much. In exchange, this allows a user to publish messages right after the other, since it just have to wait 1 second between messages. Problem is that having an rln membership being able to publish this amount of messages, is a bit of a liability for waku, and hinders scalability.
+2. A high `epoch` size (eg `240 seconds`) would allow a membership to publish `24*3600/240=360` messages a day, which is a more reasonable limit, but this won't allow a user to publish two messages one right after the other, meaning that if you publish a message, you have to way 240 seconds to publish the next one. Not practical, a no go.
+
+But what if we widen the window size, and allow multiple messages within that window?
+
+## Solution
+
+In order to fix this, we need bigger windows sizes, to smooth out particular bursts. Its fine that a user publishes 20 messages in one second, as long as in a wider window it doesn't publish more than, lets say 500. This wider window could be a day. So we could say that a membership can publish `250 msg/day`. With this we solve i) and ii) from the previous section.
+
+Some quick napkin math on how this can scale:
+- 10.000 RLN memberships
+- Each RLN membership allow to publish 250 msg/day
+- Message size of 5 kBytes
+
+Assuming a completely random distribution:
+- 10.000 * 250 = 2 500 000 messages will be published a day (at max)
+- A day has 86 400 seconds. So with a random distribution we can say that 30 msg/sec (at max)
+- 30 msg/sec * 5 kBytes/msg = 150 kBytes/sec (at max)
+- Assuming D_out=8: 150 kBytes/sec * 8 = 1.2 MBytes/sec (9.6 Mbits/sec)
+
+So while its still not possible to guarantee 100% the maximum bandwidth, if we rate limit per day we can have better guarantees. Looking at these numbers, considering a single shard, it would be feasible to serve 10.000 users considering a usage of 250 msg/day.
+
+TODO: Analysis on 95%/99% interval confidence on bandwidth given a random distribution.
+
+## TLDR
+
+- Waku should guarantee a capped bandwidth so that everyone can run a node.
+- The guarantee is a "statistical guarantee", since there is no way of enforcing a strict limit.
+- Current RLN approach is to rate limit 1 message every x seconds. A better approach would be x messages every day, which helps achieving such bandwidth limit.
+- To follow up: Variable RLN memberships. Eg. allow to chose tier 1 (100msg/day) tier 2 (200msg/day) etc.

docs/research/research-and-studies/cspell.json

@@ -6,6 +6,17 @@
         "incentivisation",
         "ipfs",
         "lightpush",
-        "waku"
+        "waku",
+        "txid",
+        "TLDR",
+        "leobago",
+        "speedtest",
+        "Mpbs",
+        "gossipsub",
+        "Mbytes",
+        "productisation",
+        "Menduist",
+        "vpavlin",
+        "disincentivise",
     ]
 }

docs/research/research-and-studies/incentivisation.md

@@ -2,17 +2,12 @@
 title: Incentivisation
 ---
 
-Waku is a family of decentralised communication protocols.
-The Waku Network (TWN) consists of independent nodes running Waku protocols.
-TWN needs incentivisation (shortened to i13n) to ensure proper node behaviour.
+Waku is a family of decentralised communication protocols. The Waku Network (TWN) consists of independent nodes running Waku protocols. TWN needs incentivisation (shortened to i13n) to ensure proper node behaviour.
 
-The goal of this document is to outline and contextualize our approach to TWN i13n.
-After providing an overview of Waku and relevant prior work,
-we focus on Waku Store - a client-server protocol for querying historical messages.
-We introduce a minimal viable addition to Store to enable i13n,
-and list research directions for future work.
+The goal of this document is to outline and contextualize our approach to TWN i13n. After providing an overview of Waku and relevant prior work, we focus on Waku Store - a client-server protocol for querying historical messages. We introduce a minimal viable addition to Store to enable i13n, and list research directions for future work.
 
 # Incentivisation in decentralised networks
+
 ## Incentivisation tools
 
 We can think of incentivisation tools as a two-by-two matrix:
@@ -25,11 +20,7 @@ In other words, there are four quadrants:
 - reputation reward: the node's reputation increases if it behaves well;
 - reputation punishment: the node's reputation decreases if it behaves badly.
 
-Reputation only works if high reputation brings tangible benefits.
-For example, if nodes chose neighbors based on reputation, low-reputation nodes miss out on potential revenue.
-Reputation scores may be local (a node assigns scores to its neighbors) or global (each node gets a uniform score).
-Global reputation in its simplest form involves a trusted third party,
-although decentralised approaches are also possible.
+Reputation only works if high reputation brings tangible benefits. For example, if nodes chose neighbors based on reputation, low-reputation nodes miss out on potential revenue. Reputation scores may be local (a node assigns scores to its neighbors) or global (each node gets a uniform score). Global reputation in its simplest form involves a trusted third party, although decentralised approaches are also possible.
 
 ## Prior work
 
@@ -37,37 +28,21 @@ We may split incentivized decentralised networks into early file-sharing, blockc
 
 ### Early P2P file-sharing
 
-Early P2P file-sharing networks employ reputation-based approaches and sticky defaults.
-For instance, the BitTorrent protocol rewards uploading peers with faster downloads.
-The download bandwidth available to a peer depends on how much it has uploaded.
-Moreover, peers share pieces of a file before having received it in whole.
-This non-monetary i13n policy has been proved to work in practice.
+Early P2P file-sharing networks employ reputation-based approaches and sticky defaults. For instance, the BitTorrent protocol rewards uploading peers with faster downloads. The download bandwidth available to a peer depends on how much it has uploaded. Moreover, peers share pieces of a file before having received it in whole. This non-monetary i13n policy has been proved to work in practice.
 
 ### Blockchains
 
-Bitcoin has introduced proof-of-work (PoW) for native monetary rewards in a P2P network.
-PoW miners are automatically assigned newly mined coins for generating blocks.
-Miners must expend physical resources to generate a block.
-If the block is invalid, these expenses are not compensated (implicit monetary punishment).
-Proof-of-stake (PoS), used in Ethereum and many other cryptocurrencies, introduces explicit monetary punishments.
-PoS validators lock up (stake) native tokens and get rewarded for validating blocks or slashed for misbehaviour.
+Bitcoin has introduced proof-of-work (PoW) for native monetary rewards in a P2P network. PoW miners are automatically assigned newly mined coins for generating blocks. Miners must expend physical resources to generate a block. If the block is invalid, these expenses are not compensated (implicit monetary punishment). Proof-of-stake (PoS), used in Ethereum and many other cryptocurrencies, introduces explicit monetary punishments. PoS validators lock up (stake) native tokens and get rewarded for validating blocks or slashed for misbehaviour.
 
 ### Decentralised storage
 
-Post-Bitcoin decentralised storage networks include Codex, Storj, Sia, Filecoin, IPFS.
-Their i13n methods combine techniques from early P2P file-sharing with blockchain-inspired reward mechanisms.
+Post-Bitcoin decentralised storage networks include Codex, Storj, Sia, Filecoin, IPFS. Their i13n methods combine techniques from early P2P file-sharing with blockchain-inspired reward mechanisms.
 
 # Waku background
 
-Waku is a [family of protocols](https://waku.org/about/architect) for a modular privacy-preserving censorship-resistant decentralised communication network.
-The backbone of Waku is the Relay protocol (and its spam-protected version [RLN-Relay](https://rfc.vac.dev/spec/17/)).
-Additionally, there are light protocols: Store, Filter, and Lightpush.
-Light protocols are also referred to as client-server protocols and request-response protocols.
+Waku is a [family of protocols](https://waku.org/about/architect) for a modular privacy-preserving censorship-resistant decentralised communication network. The backbone of Waku is the Relay protocol (and its spam-protected version [RLN-Relay](https://rfc.vac.dev/spec/17/)). Additionally, there are light protocols: Store, Filter, and Lightpush. Light protocols are also referred to as client-server protocols and request-response protocols.
 
-A server is a node running Relay and a server-side of at least one light protocol.
-A client is a node running a client-side of any of the light protocols.
-A server may sometimes be referred to as a full node, and a client as a light node.
-There is no strict definition of a full node vs a light node in Waku (see [discussion](https://github.com/waku-org/research/issues/28)).
+A server is a node running Relay and a server-side of at least one light protocol. A client is a node running a client-side of any of the light protocols. A server may sometimes be referred to as a full node, and a client as a light node. There is no strict definition of a full node vs a light node in Waku (see [discussion](https://github.com/waku-org/research/issues/28)).
 
 In light protocols, a client sends a request to a server, and a server performs some actions and returns a response:
 - [Store](https://rfc.vac.dev/spec/13/): the server responds with messages relayed that match a set of criteria;
@@ -76,18 +51,11 @@ In light protocols, a client sends a request to a server, and a server performs
 
 ## Waku i13n challenges
 
-Waku has no consensus and no native token, which brings it closer to reputation-incentivised file-sharing networks.
-As of late 2023, Waku only operates under reputation-based rewards and punishments.
-While [RLN-Relay](https://rfc.vac.dev/spec/17/) adds monetary punishments for spammers, slashing is yet to be activated.
+Waku has no consensus and no native token, which brings it closer to reputation-incentivised file-sharing networks. As of late 2023, Waku only operates under reputation-based rewards and punishments. While [RLN-Relay](https://rfc.vac.dev/spec/17/) adds monetary punishments for spammers, slashing is yet to be activated.
 
-Monetary rewards and punishments should ideally be atomically linked with the node's behaviour.
-A benefit of blockchains in this respect is that the desired behaviour of miners or validators can be verified automatically.
-Enforcing atomicity in a communication network is more challenging:
-it is non-trivial to prove that a given piece of data has been relayed.
+Monetary rewards and punishments should ideally be atomically linked with the node's behaviour. A benefit of blockchains in this respect is that the desired behaviour of miners or validators can be verified automatically. Enforcing atomicity in a communication network is more challenging: it is non-trivial to prove that a given piece of data has been relayed.
 
-Our goal is to combine monetary and reputation-based incentives for Waku.
-Monetary incentives have demonstrated their robustness in blockchains.
-We think they are necessary to scale the network beyond the initial phase when it's maintained altruistically.
+Our goal is to combine monetary and reputation-based incentives for Waku. Monetary incentives have demonstrated their robustness in blockchains. We think they are necessary to scale the network beyond the initial phase when it's maintained altruistically.
 
 ## Waku Store
 
@@ -97,8 +65,7 @@ Waku Store is a light protocol for querying historic messages that works as foll
 
 The response may be split into multiple parts, as specified by pagination parameters in `PagingInfo`.
 
-We define a _relevant_ message as a message that matches client-defined criteria (e.g., relayed within a given time frame).
-Upon receiving a request, a server should quickly send back a response containing all and only relevant messages.
+We define a _relevant_ message as a message that matches client-defined criteria (e.g., relayed within a given time frame). Upon receiving a request, a server should quickly send back a response containing all and only relevant messages.
 
 # Waku Store incentivisation
 
@@ -113,8 +80,7 @@ An incentivised Store protocol has the following extra steps:
 3. reputation
 4. results cross-checking
 
-In this document, we focus on the simplest proof-of-concept (PoC) i13n for Store.
-Compared to the fully-fledged protocol, the PoC version is simplified in the following ways:
+In this document, we focus on the simplest proof-of-concept (PoC) i13n for Store. Compared to the fully-fledged protocol, the PoC version is simplified in the following ways:
 - cost calculation is based on a common-knowledge price;
 - there is no price advertisement and no price negotiation;
 - each query is paid for in a separate transaction, `txid` acts a proof of payment;
@@ -132,12 +98,9 @@ In further subsections, we list the potential direction for future work towards
 
 ## Pricing
 
-For PoC, we assume a constant price per hour of history.
-This price and the blockchain address of the server are assumed to be common knowledge.
-This simplifies the client-server interaction, avoiding the price negotiation step.
+For PoC, we assume a constant price per hour of history. This price and the blockchain address of the server are assumed to be common knowledge. This simplifies the client-server interaction, avoiding the price negotiation step.
 
-In the future versions of the protocol, the price will be negotiated and will depend on multiple parameters,
-such as the total size of the relevant messages in the response.
+In the future versions of the protocol, the price will be negotiated and will depend on multiple parameters, such as the total size of the relevant messages in the response.
 
 ### Future work
 
@@ -148,15 +111,12 @@ such as the total size of the relevant messages in the response.
 
 ## Payment
 
-For the PoC, each request is paid for with a separate transaction.
-The transaction hash (`txid`) acts as a proof of payment.
-The server verifies the payment by ensuring that:
+For the PoC, each request is paid for with a separate transaction. The transaction hash (`txid`) acts as a proof of payment. The server verifies the payment by ensuring that:
 1. the transaction has been confirmed;
 2. the transaction is paying the proper amount to the server's account;
 3. the `txid` does not correspond to any prior response.
 
-The client gives proof of payment before it receives the response.
-Other options could be:
+The client gives proof of payment before it receives the response. Other options could be:
 1. the client pays after the fact;
 2. the client pays partly upfront and partly after the fact;
 3. a centralised third party (either trusted or semi-trusted, like a smart contract) ensures atomicity;
@@ -167,9 +127,7 @@ Our design considerations are:
 - servers are more "permanent" entities and are more likely to have long-lived identities;
 - it is more important to protect the clients's privacy than the server's privacy.
 
-In light of these criteria, we suggest that the client pays first.
-This is simpler than splitting the payment, or involving an extra atomicity-enforcing mechanism.
-Moreover, pre-payment is arguably more privacy-preserving than post-payment, which encourages servers to deanonymise clients to prevent fraud.
+In light of these criteria, we suggest that the client pays first. This is simpler than splitting the payment, or involving an extra atomicity-enforcing mechanism. Moreover, pre-payment is arguably more privacy-preserving than post-payment, which encourages servers to deanonymise clients to prevent fraud.
 
 ### Future work
 
@@ -180,8 +138,7 @@ Moreover, pre-payment is arguably more privacy-preserving than post-payment, whi
 
 ## Reputation
 
-We use reputation to discourage the server from taking the payment and not responding.
-The client keeps track of the server's reputation:
+We use reputation to discourage the server from taking the payment and not responding. The client keeps track of the server's reputation:
 - all servers start with zero reputation points;
 - if the server honours the request, it gets `+n` points;
 - if the server does not respond before a timeout, it gets `-m` points.
@@ -206,9 +163,7 @@ Design a more comprehensive reputation system:
 
 ## Results cross-checking
 
-As there is no consensus over past messages, a client may want to query multiple servers and merge their responses.
-Cross-checking helps ensure that servers are a) not censoring real messages; b) not injecting fake messages into history.
-Cross-checking is absent in PoC but may be considered later.
+As there is no consensus over past messages, a client may want to query multiple servers and merge their responses. Cross-checking helps ensure that servers are a) not censoring real messages; b) not injecting fake messages into history. Cross-checking is absent in PoC but may be considered later.
 
 ### Future work
 

docs/research/research-and-studies/maximum-bandwidth.md

@@ -0,0 +1,83 @@
+---
+title: Maximum Bandwidth for Global Adoption
+---
+
+**TLDR**: This issue aims to **set the maximum bandwidth** in `x Mbps` that each waku shard should consume so that the **maximum amount of people can run a full waku node**. It is up to https://github.com/waku-org/research/issues/22 to specify how this maximum will be enforced.
+
+**Conclusion:** Limit to `10 Mbps` each waku shard.
+
+## Introduction
+
+Waku is designed in a way that everyone should be able to run a full node on an average laptop with a residential Internet connection, at least in one shard. This will enable true decentralization and give power to the users, since they won't need to rely on third parties to send/receive messages. Professional node operators running in data centers, can of course contribute to multiple shards, but we should keep the bandwidth/hardware requirements of single shard rather low.
+
+This vision opposes the federated approach, where a few nodes requiring vast amounts of resources (cpu, memory, bandwidth) run in data centres, taking the power from the user. While federated approaches are an improvement from traditional client-server architectures, waku envisions a fully peer-to-peer architecture where anyone should be able to run a node.
+
+In order to ensure that anyone can run a node **in desktop**, there are two main **limiting factors**:
+1. Bandwidth consumption in Mbps
+2. CPU/memory resources (mainly limited by RLN proof verification)
+
+This issue focuses on i) bandwidth consumption and https://github.com/waku-org/research/issues/30 on ii) CPU/memory resources. Note that on https://github.com/waku-org/research/issues/23 an analysis on the impact on RLN was already made, but wasn't focused on scalability. Said issues do.
+
+In https://github.com/waku-org/research/issues/22 we discussed **why** and **how** to limit the maximum bandwidth per shard, but we haven't come up with a specific number in Mbps. **This issue i) presents data from the available bandwidth at different locations and ii) suggests a maximum bandwidth in Mbps that waku should enforce**.
+
+## Bandwidth availability and usage
+
+The following tables show:
+- Table [1] The Q25, Q75 and average bandwidth (upload/download) in Mbps available on different continents. Raw data is available [here](https://www.measurementlab.net/data/) and credits to [@leobago](https://github.com/leobago) for the summarized version. Note: The below numbers were rounded to the nearest integer.
+- Table [2] The median global bandwidth (upload/download) in Mbps, taken from [speedtest](https://www.speedtest.net/global-index) (accessed 12 Oct 2023). 
+- Table [3] Download bandwidth requirements in Mbps for Netflix video streaming, [source](https://www.comparethemarket.com/broadband/content/broadband-for-streaming/).
+
+|    *Table [1]*             | Download (Mbps) |            |        | Upload (Mbps) |            |        |
+|------------------|-----------------|------------|--------|---------------|------------|--------|
+|                  |   **Q25**           |   **Average**  |  **Q75**  |   **Q25**         |   **Average**  |   **Q75**  |
+|   North-America  |   58            |   107      |   137  |   38          |   68       |   85   |
+|   South-America  |   21            |   54       |   72   |   13          |   33       |   44   |
+|   Europe         |   49            |   93       |   119  |   30          |   56       |   72   |
+|   Asia           |   23            |   53       |   71   |   15          |   37       |   50   |
+|   Oceania        |   44            |   84       |   108  |   27          |   50       |   63   |
+|   Africa         |   12            |   26       |   33   |   7           |   17       |   22   |
+
+|   *Table [2]*     | Median Download (Mbps) | Median Upload (Mbps) |
+|--------|------------------------|----------------------|
+| Global | 83                     | 38                   |
+
+| *Table [3]* **Video resolution** | **Recommended Bandwidth** | 
+|----------------------|---------------------------|
+| HD                   | 3 Mbps                    | 
+| Full HD              | 5 Mbps                    | 
+| 4K/UHD               | 15 Mbps                   | 
+
+## Selecting a maximum bandwidth
+
+With the above data, we should be informed to take a decision on the maximum bandwidth that we should enforce per shard. With this number, we will apply the techniques explained in https://github.com/waku-org/research/issues/22 to ensure (with some statistical confidence) that the bandwidth won't exceed that number.
+
+The **trade-off is clear**:
+- We **enforce a low bandwidth**: more people can run full waku nodes, overall network throughput is less, network decentralization is easier, gives power to the user as its fully sovereign.
+- We **don't enforce a low bandwidth**: not possible to run full waku nodes in laptops acting as a centralization force, nodes are run by few professional operators in data centers, waku users just use light clients, network throughput can scale way easier, federated approach.
+
+So it's about where to draw this line.
+
+Points to take into account:
+- **Relay contributes to bandwidth the most**: Relay is the protocol that mostly contributes to bandwidth usage, and it can't choose to allocate fewer bandwidth resources like other protocols (eg `store` can choose to provide less resources and it will work). In other words, the network sets the relay bandwidth requirements, and if the node can't meet them, it just wont work.
+- **Upload and download bandwidth are the same**: Due to how gossipsub works, and hence `relay`, the bandwidth consumption is symmetric, meaning that upload and download bandwidth is the same. This is because of `D` and the reciprocity of the connections, meaning that one node upload is another download.
+- **Nodes not meeting requirements can use light clients**. Note that nodes not meeting the bandwidth requirements can still use waku, but they will have to use light protocols, which are a great alternative, especially on mobile, but with some drawbacks (trust assumptions, less reliability, etc)
+- **Waku can't take all the bandwidth:** Waku is meant to be used in conjunction with other services, so it shouldn't consume all the existing bandwidth. If Waku consumes `x Mbps` and someone bandwidth is `x Mpbs`, the UX won't be good.
+- **Compare with existing well-known services:** As shown in *Table [3]*, Netflix 4K video streaming takes 15Mbps, so that is an order of magnitude to take into account.
+
+Coming up with a number:
+- Lowest average download speed across continents is Africa (26 Mbps)
+- Lowest average upload speed across continents is Africa (17 Mbps)
+- Since in waku the bandwidth consumption is symmetric, we are limited by the lowest (17 Mpbs)
+- However waku should not consume all bandwidth, leaving some room for other applications.
+- We could set 10 Mbps, which is between Full HD video and 4K.
+- With 10Mbps the % of average bandwidth waku will consume is:
+  - North-America 9 %
+  - South-America 18 %
+  - Europe  11 %
+  - Asia 18 %
+  - Oceania 12 %
+  - Africa 38 %
+
+**Conclusion:** Limit to `10 Mbps` each waku shard. How? Not trivial, see https://github.com/waku-org/research/issues/22#issuecomment-1727795042
+
+*Note:* This number is not set in stone and is subject to modifications, but it will most likely stay in the same order of magnitude if changed.

docs/research/research-and-studies/message-propagation.md

@@ -0,0 +1,67 @@
+---
+title: Message Propagation Times With Waku-RLN
+---
+
+**TLDR**: We present the results of 1000 `nwaku` nodes running `rln` using different message sizes, in a real network with bandwidth limitations and network delays. The goal is to study the message propagation delay distribution, and how it's affected by i) rln and ii) message size in a real environment. We observe that for messages of `10kB` the average end-to-end propagation delay is `508 ms`. We can also observe that the message propagation delays are severely affected when increasing the message size, which indicates that it is not a good idea to use waku for messages of eg. `500kB`. See simulation parameters.
+
+## Introduction
+
+Waku uses [relay](https://rfc.vac.dev/spec/11/) as a routing protocol, which is an adaptation of [gossipsub](https://arxiv.org/pdf/2007.02754.pdf). It routes messages following a publisher/subscriber architecture, where nodes can publish messages or subscribe to topics. If message `m` is published to topic `t`, all `i` nodes `n_1...n_i` subscribed to `t` will get `m`. The `relay` protocol ensures that every node gets the messages of the topics it is subscribed to.
+
+However, since `relay` works in a decentralized manner, all nodes contribute to the gossiping of a message, until it has successfully reached all the interested nodes (subscribed to it). This means that a message can travel multiple hops until it reaches all nodes. The amount of hops determines the **message propagation time**, which is measured as the **time difference of when the node published the message and when another node received**.
+
+**This issue aims to go from theory to practice, by i) understanding message propagation times in theory and ii) presenting nwaku simulation results in an end-to-end setup with rln, with real message propagation times**.
+
+## Theory
+
+Let's start with **message propagation times in theory**. On a high level, it depends on:
+- The gossipsub [configuration](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters), being `D` one of the most important parameters. This sets the hops that a message will travel to reach all nodes. Higher `D`, less hops, less delay. Note that a higher `D` implies more bandwidth consumption.
+- The node. Different nodes will see different propagation times, because a message can travel different paths. A node connected directly to the publisher (1 hop) will see lower propagation times than other nodes further away.
+- Individual propagation times. Since a message can travel multiple hops to reach its destination, each hop adds a contribution to the overall message propagation time. This individual propagation time depends on the characteristics on the nodes involved in the connections.
+
+In a D-regular graph, like the one formed by waku nodes around a topic, the maximum amount of hops that a message can travel to reach all nodes can be calculated as `ceil(log(total_nodes)/log(D))`. For example, with log(1000)/log(6) = 3.85 = 4. So in a network with 1000 nodes and `D=6`, no matter which node publishes the message, in 4 hops it will reach all the nodes.
+
+Notice the **"worst case"** since some nodes might be directly connected to the publisher, so they will get the message in just 1 hop.
+
+But how long does it take to jump each hop? It depends on:
+- The latency between nodes. Can be measured as the time to respond to a ping.
+- The size of the messages. The bigger the message, the more time it takes to transmit.
+- Nodes bandwidth. Sender upload bandwidth and receiver download bandwidth. More important when using big message sizes.
+- Message validation time. When each node receives a message, it applies some validation to decide if the message is further gossiped or not. In the case of waku, this is RLN ([paper](https://arxiv.org/pdf/2207.00116.pdf), [rfc](https://rfc.vac.dev/spec/32/))
+
+Assuming a message `m` that travels 4 hops from node `n1` (publisher) to `n5` (subscriber) we can calculate the message propagation time `mpt=ipt_1+ipt_2+ipt_3+ipt_4` where `ipt` is the individual propagation time between each node in the chain.
+
+However, specific message propagation times are useless, we need average times under specific conditions. And **for this, we need simulations**.
+
+## Simulations
+
+Using [shadow](https://shadow.github.io/docs/guide/shadow.html) simulator, we have developed a [tool](https://github.com/waku-org/research/tree/master/rln-delay-simulations) that allows to simulate message propagation delays of `nwaku` (using a slightly modified [branch](https://github.com/waku-org/nwaku/compare/master...simulations), mainly to instrument it with tools to measure the times + starting from an already connected mesh. Thanks [@Menduist](https://github.com/menduist) for the help. Note that running this simulation requires a significant amount of resources, done with 256 GB of RAM.
+
+The configuration of the simulation is (see [config](https://github.com/waku-org/research/blob/master/rln-delay-simulations/shadow.yaml)):
+- `latency=100ms`. Average latency in our current waku network. Thanks [@vpavlin](https://github.com/vpavlin) for the measurements. See [this](https://grafana.infra.status.im/d/b819dbfe-acb6-4086-8736-578ca148d7cd/waku-networkmonitor-v2?orgId=1&refresh=30s&viewPanel=12) for live data.
+- `down_bandwidth=83Mbps`, `up_bandwidth=38Mbps`. As shown in [Table 2](https://github.com/waku-org/research/issues/31) that's the worldwide median speed.
+- `D=6`, which is the current `nwaku` [configuration](https://github.com/waku-org/nwaku/blob/v0.21.0/waku/waku_relay/protocol.nim#L73-L78).
+- `nodes=1000`. Amount of nodes used in the simulation
+- `nwaku` was used with a minor [modification](https://github.com/waku-org/nwaku/compare/master...simulations)
+- A total of `10` messages were published, that led to `9990` received messages.
+- Since `shadow` **doesn't take into account CPU times** ([by now](https://github.com/shadow/shadow/discussions/1675#discussioncomment-7342812)), we simulate it with `sleepAsync` as per https://github.com/waku-org/research/issues/23 findings. `0.012 seconds` for proof verification and `0.15 seconds` for proof generation.
+
+## Results
+
+The following figure shows the **message propagation time with real simulations**, showing the distribution in a network with the above configuration with three different message sizes: `10kB`, `100kB`, `500kB`. Note that the whiskers indicate the best/worst values and the box contains P25 to P75 values. Average `mu` and P95 are also shown. Raw data [here](https://github.com/waku-org/research/tree/master/rln-delay-simulations/raw).
+
+![message-latency-distribution](imgs/message-latencies-distribution.png)
+
+**Important note**. The first messages sent in the simulations are omitted, since they show an abnormal propagation delay that doesn't reflect reality. This is due to how flow control works in TCP, where right after connection, the sender node has no idea of the "bandwidth" of the receiver node, so it will start sending packages at a lower rate. This translates into high transmission times, and it's more pronounced when dealing with big message sizes.
+
+In other words, in a 100Mpbs link, 100Mbits won't be sent in 1 second, or at least not a the beginning, when the node is slowly increasing the rate until based on ACK/NACK ratio. For more information about this, this is explained in [here](https://www.youtube.com/watch?v=vb_wjh_nAmo).
+
+**Conclusions:**
+- Using small messages `10kB` the **average propagation delay is `508 ms`**, quite reasonable for applications using waku. The variance is acceptable, with 95% of the messages arriving in `<627 ms`.
+- When using a size of `10kB` we can see that the best case propagation delay is `263 ms`. This corresponds to the nodes that are just 1 hop from the publisher. The proof generation time `0.15 seconds` affects the most, where the rest is the inter-node latency and the transmission of the message itself.
+- We can see that the **message propagation delay increases with big messages**, `100kB` and `500kB`. So its **probably not a good idea to use waku for such big messages**. Note that these simulations had 1000 nodes, so if we scale it to 10000 or beyond, propagation times would be worse.
+- Best case propagation time (lower part of the whisker) is quite similar in all cases. This is because it corresponds to the node that is just 1 hop away from the publisher.
+
+**Future work**:
+- Current waku `D` [values](https://github.com/waku-org/nwaku/blob/v0.21.0/waku/waku_relay/protocol.nim#L73-L78) (average of 6 ranging from 4 to 12) have a huge impact on the bandwidth that a node consumes. Are we willing to lower D in order to reduce bandwidth but increase message propagation times?
+- Since `shadow` doesn't take CPU time into account, it's currently simulated for rln, which should be the biggest bottleneck. Once `shadow` has [this feature](https://github.com/shadow/shadow/discussions/1675#discussioncomment-7342812) times would be more accurate.

docs/research/research-and-studies/rln-key-benchmarks.md

@@ -0,0 +1,67 @@
+---
+title: RLN Key Benchmarks
+---
+
+## Introduction
+
+Since RLN has been chosen as the spamming protection mechanism for waku, we must understand the practical implications of using it. This issue explains the main differences between `relay` and `rln-relay` and gives some benchmarks after running simulations using `waku-simulator`, in a network with the following characteristics:
+- 100 nwaku nodes, each one with a valid rln membership and publishing a message every 10 seconds to a common topic.
+- rln contract deployed in Ethereum Sepolia
+- 10.000 memberships registered in the contract
+- pure relay (store and light protocols disabled)
+
+The main deltas `rln` vs `rln-relay` are:
+- New `proof ` field in `WakuMessage` containing 384 extra bytes. This field must be generated and attached to each message.
+- New validator, that uses `proof` to `Accept` or `Reject` the message. The proof has to be verified.
+- New dependency on a blockchain, Ethereum, or any EVM chain, to keep track of the members allowed to publish.
+
+But what are the practical implications of these?
+
+## TLDR:
+- Proof generation is constant-ish. 0.15 second for each proof
+- Proof verification is constant-ish, 0.012 seconds. In a network with 10k nodes and D=6 this would add an overhead delay of 0.06 seconds.
+- Gossipsub scoring drops connections from spammer peers, which acts as the punishment (instead of slashing). Validated in the simulation.
+- Rln doesn't have any impact on memory consumption.
+
+## Proof generation times
+
+Seems that proof generation times stay constant no matter the size of the message. In the following simulation it was increased from: `1kB`, `10kB`, `50kB`, `150kB`. On average it takes `0.15 seconds` to calculate the message proof. This means that when a node wants to send a message, it will need to spend this time generating the proof. It seems very reasonable and it actually acts as a mini proof of work, where a consumer computer won't be able to publish a really high number of messages per second.
+
+![proof-generation-times](imgs/proof-generation-times.png)
+
+## Proof verification times
+
+On the other hand, rln also adds an overhead in the gossipsub validation process. On average it takes `0.012 seconds` to verify the proof. It seems that when we increase the message size, validation time seems to increase a bit, which can be for any other reason besides rln itself (eg deserializing the message might take longer).
+
+This number seems reasonable and shouldn't affect that much the average delay of a message. Assuming a d-regular graph, with `10k` nodes and `D=6`, we can have up to  `log(total_nodes)/log(D)=5` hops. So in the worst case, rln will add a network latency of `0.012*5 = 0.06 seconds`
+
+![proof-verification-times](imgs/proof-verification-times.png)
+
+## Spam protection
+
+For the initial release of RLN, slashing won't be implemented and it still remains unclear if it will be implemented in the future. Luckily, even if slashing is not implemented rln can be used to detect spam and punish the sender off-chain (instead of slashing an onchain collateral). This is done with gossipsub scoring.
+
+In the following simulation, we can see `100` nwaku interconnected nodes, where one of them suddenly starts spamming the network with multiple valid rln messages `3000 messages/minute`. Since its rate limited to 1msg/10 seconds, we can see that in almost no time, every node in the network disconnects from the spammer peer (see red node), leaving it with `0 peers`, which disincentivise such attacks and without requiring a financial slashing.
+
+![connected-peers](imgs/connected-peers.png)
+
+## RLN tree sync
+
+Using RLN implies that waku should now piggyback on a blockchain (the case study uses Ethereum Sepolia) and has to stay up to date with the latest events emitted by the rln smart contract. These events are used to locally construct a tree that contains all members allowed to create valid proofs to send messages. Some numbers:
+- A tree with 10k members takes `2Mbytes` of space. Negligible.
+- A tree with 10k members takes `<4 minutes to synchronize. Assumable since it's done just once.
+- With a block range of 5000 blocks for each request, we would need `520 requests` to synchronize 1 year of historical data from the tree. Assumable since most of the free endpoints out there allow 100k/day.
+
+## Performance relay vs. rln-relay
+
+Same simulation with 100 nodes was executed `with rln` and `without rln`:
+- Memory consumption is almost identical
+
+**with rln**
+![with-rln](imgs/with-rln.png)
+
+**without rln**
+![without-rln](imgs/without-rln.png)
+
+- Couldn't capture cpu metrics
+- Minor differences in messages per seconds is due to injection technique, nothing related to rln itself.

fetch-content.js

@@ -63,17 +63,25 @@ const repositories = [
     {
         baseUrl: 'https://api.github.com/repos/waku-org/nwaku/contents/docs/benchmarks',
         baseSavePath: '/docs/research/benchmarks/',
-        prefixToRemove: 'docs/benchmarks/'
+        prefixToRemove: "docs/benchmarks/",
+        categoryFileContent: "{ \"label\": \"Benchmarks\", \"collapsed\": false }"
     },
     {
         baseUrl: 'https://api.github.com/repos/waku-org/research/contents/docs',
         baseSavePath: '/docs/research/research-and-studies/',
-        prefixToRemove: 'docs/'
+        prefixToRemove: "docs/",
+        categoryFileContent: "{ \"label\": \"Research and Studies\", \"collapsed\": false }"
     }
 ];
 
-fs.rmSync('docs/research/', { recursive: true, force: true });
+repositories.forEach(repo => {
+    fs.rmSync(path.join(__dirname, repo.baseSavePath), { recursive: true, force: true });
+});
 
 repositories.forEach(repo => {
-    fetchDirectoryContents(repo.baseUrl, repo.baseSavePath, repo.prefixToRemove);
-});
+    fetchDirectoryContents(repo.baseUrl, repo.baseSavePath, repo.prefixToRemove, repo.categoryFileContent).then(() => {
+        const dir = path.join(__dirname, repo.baseSavePath);
+        fs.mkdirSync(dir, { recursive: true });
+        fs.writeFileSync(path.join(dir, "_category_.json"), repo.categoryFileContent);
+    });
+});

sidebars.js

@@ -87,18 +87,10 @@ const sidebars = {
 	],
 	research: [
 		{
-			type: "category",
-			label: "Research and Studies",
-			collapsed: false,
-			items: ["research/research-and-studies/incentivisation"],
-		},
-		{
-			type: "category",
-			label: "Nwaku Benchmarks",
-			collapsed: false,
-			items: ["research/benchmarks/postgres-adoption"],
+			type: 'autogenerated',
+			dirName: 'research', // '.' means the current docs folder
 		},
 	],
 };
 
-module.exports = sidebars;
+module.exports = sidebars;