Add waku-simulator book (#65)

.gitignore

@@ -1,3 +1,4 @@
 **/.DS_Store
 *.env
 !wakusim.env
+book

book.toml

@@ -0,0 +1,7 @@
+[book]
+authors = ["waku-org"]
+language = "en"
+multilingual = false
+src = "src"
+title = "Waku Simulator Book"
+description = "A brief description of your book."

src/SUMMARY.md

@@ -0,0 +1,9 @@
+# Summary
+
+- [Introduction to Waku Simulator](./intro-waku-simulator.md)
+- [Deploy a waku network](./deploy-waku-network.md)
+- [Inject traffic](./inject-traffic.md)
+- [Connect external full node](./connect-full-node.md)
+- [Connect external spam node](./connect-spam-node.md)
+- [Connect external light node](./connect-light-node.md)
+- [Register memberships](./register-memberships.md)

src/connect-full-node.md

@@ -0,0 +1,51 @@
+# Connect external full node
+
+
+If you want to connect to the existing waku network a node with some custom configuration. Perhaps a different image or some other configuration, you can do it as follows. Bear in mind that if this node has other configuration (eg `rln-relay-epoch-sec` or `rln-relay-user-message-limit`) then it won’t behave properly.
+
+- ⚠️set your own `staticnode`
+
+```bash
+docker run -it --network waku-simulator_simulation quay.io/wakuorg/nwaku-pr:2759-rln-v2 \
+      --relay=true \
+      --rln-relay=true \
+      --rln-relay-dynamic=true \
+      --rln-relay-eth-client-address=http://foundry:8545 \
+      --rln-relay-eth-contract-address=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 \
+      --rln-relay-epoch-sec=1 \
+      --rln-relay-user-message-limit=1 \
+      --log-level=DEBUG \
+      --staticnode=/ip4/10.2.0.16/tcp/60000/p2p/16Uiu2HAmAA99YfoLitSXgY1bHaqjaTKhyrU4M4y3D1rVj1bmcgL8 \
+      --pubsub-topic=/waku/2/rs/66/0 \
+      --cluster-id=66
+```
+
+You can for example try to connect a node running in a different `cluster-id` or other weird scenarios.
+
+You can also try to connect multiple nodes with a loop. Note the `&`. Remember to kill the new nodes once you are done.
+
+```bash
+for i in {1..5}; do
+    docker run -it --network waku-simulator_simulation quay.io/wakuorg/nwaku-pr:2759-rln-v2 \
+      --relay=true \
+      --rln-relay=true \
+      --rln-relay-dynamic=true \
+      --rln-relay-eth-client-address=http://foundry:8545 \
+      --rln-relay-eth-contract-address=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 \
+      --rln-relay-epoch-sec=1 \
+      --rln-relay-user-message-limit=1 \
+      --log-level=DEBUG \
+      --staticnode=/ip4/10.2.0.16/tcp/60000/p2p/16Uiu2HAmAA99YfoLitSXgY1bHaqjaTKhyrU4M4y3D1rVj1bmcgL8 \
+      --pubsub-topic=/waku/2/rs/66/0 \
+      --cluster-id=66 &
+done
+```
+
+🎯**Goals**:
+
+- Connect a different nodes(s) to the network for some ad hoc test.
+- See how the network reacts to a node with different configuration.
+
+👀**Observability**:
+
+- Check the new node logs, ensuring the behaviour matches the expected.

src/connect-light-node.md

@@ -0,0 +1,57 @@
+# Connect external light node
+
+By using [go-waku-light](https://github.com/alrevuelta/go-waku-light), you can connect one or multiple light clients to the network. This utility can be configured to send messages at a given rate using a given peer as `ligh-push`. It will register a RLN membership at startup. Bear in mind that it should be configured with the same contract and `user-message-limit` as the waku nodes. You should modify the `lightpush-peer`.
+
+Note that if you spin up multiple service like this using the same `priv-key` some of the transactions registering the RLN membership may fail due to the nonce being repeated. This can be fixed by using multiple keys or waiting for the registration to be completed before spinning up the next process.
+
+- ⚠️ change `lightpush-peer` to the node you wish. Note that the multiaddress is logged by every peer at startup.
+
+```jsx
+docker run --network waku-simulator_simulation alrevuelta/go-waku-light:07b8f32 \
+--eth-endpoint=http://foundry:8545 \
+--contract-address=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 \
+send-messages-loop \
+--priv-key=ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 \
+--user-message-limit=1 \
+--message="light client sending a rln message" \
+--content-topic=/basic2/1/test/proto \
+--pubsub-topic=/waku/2/rs/66/0 \
+--cluster-id=66 \
+--lightpush-peer=/ip4/10.2.0.16/tcp/60000/p2p/16Uiu2HAmAA99YfoLitSXgY1bHaqjaTKhyrU4M4y3D1rVj1bmcgL8 \
+--message-every-secs=5
+```
+
+Note that in some examples, it could be interesting to run multiple instances, either in parallel or one after the other. For example if you set `amount-message-to-send=1` this will send just 1 message and exit. You can for example run this 100 times, where a fresh RLN membership will be created on every run, create a new peerId, send a message and exit.
+
+```bash
+for i in {1..5}; do
+    docker run --rm --network waku-simulator_simulation alrevuelta/go-waku-light:07b8f32 \
+    --eth-endpoint=http://foundry:8545 \
+    --contract-address=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 \
+    send-messages-loop \
+    --priv-key=ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 \
+    --user-message-limit=1 \
+    --message="light client sending a rln message" \
+    --content-topic=/basic2/1/test/proto \
+    --pubsub-topic=/waku/2/rs/66/0 \
+    --cluster-id=66 \
+    --lightpush-peer=/ip4/10.2.0.16/tcp/60000/p2p/16Uiu2HAm6a4kUT7YutsbwgQcmWw5VLzN3zj1StwiBVf2LUH9kb4A \
+    --message-every-secs=5 \
+    --amount-message-to-send=1
+    
+    if [ $? -ne 0 ]; then
+        echo "Command failed at iteration $i"
+        break
+    fi
+done
+```
+
+🎯**Goals**:
+
+- Tests lightpush end to end, where proofs are fetched directly from the contract
+
+👀**Observability**:
+
+- Check the logs of the node you provided as `lightpush-peer`.
+- Check grafana metrics at `localhost:3001`.
+- Check that the RLN membership was registered in the block explorer `localhost:300`.

src/connect-spam-node.md

@@ -0,0 +1,16 @@
+# Connect external spam node
+
+
+You can connect a node to the network that spams the nodes, sending messaging exceeding its rate limit.
+
+```bash
+# TODO: Wait until we have waku-spammer
+```
+
+🎯**Goals**:
+
+- TODO:
+
+👀**Observability**:
+
+- TODO:

src/deploy-waku-network.md

@@ -0,0 +1,60 @@
+# Deploy a waku network
+
+The network can be deployed in a few commands, and requires `docker` and `docker-compose`. Some of the configuration is exposed via env flags, but if you are missing some, PRs are accepted.
+
+Some of the most important parameters are:
+
+- `NWAKU_IMAGE` Docker image of nwaku that all nodes will run
+- `NUM_NWAKU_NODES` Amount of nwaku nodes
+- `RLN_RELAY_EPOCH_SEC` and `RLN_RELAY_MSG_LIMIT` configure the RLNv2 parameter, specifying the amount of messages that are allowed per unit of time.
+- `TRAFFIC_DELAY_SECONDS` and `MSG_SIZE_KBYTES` are used to inject traffic via the rest API into the network.
+
+```bash
+export NWAKU_IMAGE=quay.io/wakuorg/nwaku-pr:2759-rln-v2
+export NUM_NWAKU_NODES=5
+export RLN_RELAY_EPOCH_SEC=1
+export RLN_RELAY_MSG_LIMIT=1
+
+export TRAFFIC_DELAY_SECONDS=15
+export MSG_SIZE_KBYTES=10
+
+export PRIVATE_KEY=0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
+export ETH_FROM=0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266
+```
+
+Once configured, start all containers:
+
+```bash
+docker-compose --compatibility up -d
+```
+
+After couple of minutes, everything should be running at:
+
+- `http://localhost:3000` Block explorer
+- `http://localhost:3001` Grafana metrics
+
+For greater observability, one can access each node logs as follows:
+
+```bash
+docker logs waku-simulator_nwaku_1
+docker logs waku-simulator_nwaku_2
+```
+
+Or if you want to follow the logs
+
+```bash
+docker logs waku-simulator_nwaku_1 --follow
+```
+
+Once the network of nwaku nodes is up and running we can use it to perform different tests, connecting other nodes that we fully control with some specific characteristics. This ranges from connecting spammer nodes, light clients, and in the future unsynced nodes, etc.
+
+
+Now that we have the network deployed we can use it. Hereunder we describe how to use the network deployed by `waku-simulator` to perform end-to-end tests of any desired feature. We focus on the following ones:
+
+- Inject traffic:
+- Connect external full node:
+- Connect external spammer node:
+- Connect external light node:
+- Register memberships:
+
+⚠️ For every use case, ensure that your node is configured in the same way as the rest of the nodes, otherwise message may be lost. Note that it can be also an intended test, seeing how the network reacts to other nodes connecting to it.

src/inject-traffic.md

@@ -0,0 +1,24 @@
+# Inject traffic
+
+
+In order to inject traffic into the network, we can use the REST API of each nwaku node. We have a simple dockerized script in [rest-traffic](https://github.com/alrevuelta/rest-traffic), that can perform this task. In the following command we run a docker container, connected to the waku-simulator network. This script will inject a message every `delay-seconds` with a size of `msg-size-kbytes` into a given `pubsub-topic`. Note that in `multiple-nodes` you can configure the nodes that will publish messages, where `[1..5]` will publish to node 1, 2, 3, 4, 5. You can publish to a single node (eg node 1) by using `[1..1]`.
+
+```jsx
+docker run -it --network waku-simulator_simulation alrevuelta/rest-traffic:d936446 \
+--delay-seconds=10 \
+--msg-size-kbytes=5 \
+--pubsub-topic=/waku/2/rs/66/0 \
+--multiple-nodes="http://waku-simulator_nwaku_[1..5]:8645"
+```
+
+Note that the REST API doesn’t allow to publish messages exceeding the rate limit, so this tool can’t be used to tests beyond the rate limits.
+
+🎯**Goals**:
+
+- Test message publishing via the REST API and that they are valid across the network.
+- Test the network under different message load: rates and sizes.
+
+👀**Observability**:
+
+- Check the logs of each `nwaku` node, verifying the messages are correctly received via the API and forwarded to other nodes.
+- Check grafana dashboard `localhost:3001` ”RLN Valid Messages Total” increases as expected.

src/intro-waku-simulator.md

@@ -0,0 +1,19 @@
+# Introduction to Waku Simulator
+
+The [waku-simulator](https://github.com/waku-org/waku-simulator) tool allows simulating a waku network with a set of interconnected [nwaku](https://github.com/waku-org/nwaku) nodes with the following features:
+
+- Configurable amount of nodes. Limits depends on the machine and upper bounded at around 200.
+- Runs in a single machine, using `docker-compose` to orchestrate the containers.
+- It uses discv5 for peer discovery, using a common bootstrap node.
+- It runs a custom ad hoc network, isolated from the existing waku networks.
+- It uses a freshly deployed private blockchain, with full control over it and minimum state to track.
+- It deploys an RLN contract in the said private blockchain and configures it to be used by all nodes.
+- It registers an RLN membership for each node in the network, configuring it in the node to publish valid messages.
+- It exposes each node’s API, so that it can be used to inject traffic into the network.
+- Simple to run. Everything is automated. Requires two commands to run.
+
+The main goals of `waku-simulator` includes but is not limited to:
+* Test new features in an end to end setup with multiple nodes.
+* Use as a long-lived running network on latest master, to anticipate to breaking changes.
+* Explore waku's limits by using different loads and configurations.
+* Offer a tool to debug problems in a controled and easy to replicate environment.

src/register-memberships.md

@@ -0,0 +1,26 @@
+# Register memberships
+
+
+The [go-waku-light](https://github.com/alrevuelta/go-waku-light) tool can be used as well to register multiple RLN memberships. This can be useful to stress test the nodes, forcing a large amount of memberships. Set `amount` to the amount of memberships that you want to register. Note that it takes some time, since memberships are registered one after the other. You can spin up multiple service like this, but in that case you must provide different `priv-key` to each. Note that this memberships are kind of thrown away and not used to send messages.
+
+```jsx
+docker run --network waku-simulator_simulation alrevuelta/go-waku-light:07b8f32 \
+--eth-endpoint=http://foundry:8545 \
+--contract-address=0xCf7Ed3AccA5a467e9e704C703E8D87F634fB0Fc9 \
+register \
+--priv-key=ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 \
+--user-message-limit=1 \
+--amount=100
+
+```
+
+🎯**Goals**:
+
+- Tests how the network reacts to a massive amount of memberships.
+- Tests that nodes pick-up new memberships in real time and match.
+
+👀**Observability**:
+
+- Check the RLN memberships being registered in the block explorer `localhost:3000`.
+- Check in grafana `localhost:3001` the metric “RLN Registered Memberships” and verify the nodes are picking up the new registrations.
+