Move and update Readme.md (#9)

.gitignore

@@ -19,3 +19,7 @@
 *.la
 *.exe
 *.dll
+
+# Ignore simulation generated metrics files
+/metrics/prometheus
+/metrics/waku-sim-all-nodes-grafana-dashboard.json

README.md

@@ -1,2 +1,134 @@
 # nim-waku
-Waku node and protocol.
+## Introduction
+The nim-waku repository holds a Nim implementation of the [Waku protocol](https://specs.vac.dev/waku/waku.html) and a cli application `wakunode` that allows you to run a Waku
+enabled node from command line.
+
+The Waku specification is still in draft and thus this implementation will
+change accordingly. For supported specification details see [here](#spec-support).
+
+Additionally the original Whisper (EIP-627) protocol can also be enabled as can
+an experimental Whisper - Waku bridging option.
+
+The underlying transport protocol is [rlpx + devp2p](https://github.com/ethereum/devp2p/blob/master/rlpx.md) and the [nim-eth](https://github.com/status-im/nim-eth) implementation is used.
+
+This repository is also a place for experimenting with possible future versions
+of Waku such as replacing the transport protocol with libp2p.
+
+## How to Build & Run
+
+### Prerequisites
+
+* GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.
+* PCRE
+
+More information on the installation of these can be found [here](https://github.com/status-im/nimbus#prerequisites).
+
+### Wakunode
+
+```bash
+# The first `make` invocation will update all Git submodules.
+# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
+make wakunode
+
+# See available command line options
+./build/wakunode --help
+
+# Connect the client directly with the Status test fleet
+./build/wakunode --log-level:debug --discovery:off --fleet:test --log-metrics
+```
+
+### Waku Protocol Test Suite
+
+```bash
+# Run all the tests
+make test
+```
+
+You can also run a specific test (and alter compile options as you want):
+```bash
+# Get a shell with the right environment variables set
+./env.sh bash
+# Run a specific test
+nim c -r ./tests/v1/test_waku_connect.nim
+```
+
+### Waku Quick Simulation
+One can set up several nodes, get them connected and then instruct them via the
+JSON-RPC interface. This can be done via e.g. web3.js, nim-web3 (needs to be
+updated) or simply curl your way out.
+
+The JSON-RPC interface is currently the same as the one of Whisper. The only
+difference is the addition of broadcasting the topics interest when a filter
+with a certain set of topics is subcribed.
+
+The quick simulation uses this approach, `start_network` launches a set of
+`wakunode`s, and `quicksim` instructs the nodes through RPC calls.
+
+Example of how to build and run:
+```bash
+# Build wakunode + quicksim with metrics enabled
+make NIMFLAGS="-d:insecure" wakusim
+
+# Start the simulation nodes, this currently requires multitail to be installed
+./build/start_network --topology:FullMesh --amount:6 --test-node-peers:2
+# In another shell run
+./build/quicksim
+```
+
+The `start_network` tool will also provide a `prometheus.yml` with targets
+set to all simulation nodes that are started. This way you can easily start
+prometheus with this config, e.g.:
+
+```bash
+cd ./metrics/prometheus
+prometheus
+```
+
+A Grafana dashboard containing the example dashboard for each simulation node
+is also generated and can be imported in case you have Grafana running.
+This dashboard can be found at `./metrics/waku-sim-all-nodes-grafana-dashboard.json`
+
+To read more details about metrics, see [next](#using-metrics) section.
+
+## Using Metrics
+
+Metrics are available for valid envelopes and dropped envelopes.
+
+To compile in an HTTP endpoint for accessing the metrics we need to provide the
+`insecure` flag:
+```bash
+make NIMFLAGS="-d:insecure" wakunode
+./build/wakunode --metrics-server
+```
+
+Ensure your Prometheus config `prometheus.yml` contains the targets you care about, e.g.:
+
+```
+scrape_configs:
+  - job_name: "waku"
+    static_configs:
+      - targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']
+```
+
+For visualisation, similar steps can be used as is written down for Nimbus
+[here](https://github.com/status-im/nimbus#metric-visualisation).
+
+There is a similar example dashboard that includes visualisation of the
+envelopes available at `waku/node/v1/examples/waku-grafana-dashboard.json`.
+
+## Spec support
+
+*This section last updated April 21, 2020*
+
+This client of Waku is spec compliant with [Waku spec v1.0](https://specs.vac.dev/waku/waku.html).
+
+It doesn't yet implement the following recommended features:
+- No support for rate limiting
+- No support for DNS discovery to find Waku nodes
+- It doesn't disconnect a peer if it receives a message before a Status message
+- No support for negotiation with peer supporting multiple versions via Devp2p capabilities in `Hello` packet
+
+Additionally it makes the following choices:
+- It doesn't send message confirmations
+- It has partial support for accounting:
+  - Accounting of total resource usage and total circulated envelopes is done through metrics But no accounting is done for individual peers.

waku/node/README.md

@@ -1,5 +0,0 @@
-# Waku Node
-
-TODO.
-
-See README in `v1` folder for instructions on how to run a node.

waku/node/v1/README.md

@@ -1,108 +0,0 @@
-# Introduction
-`wakunode` is a cli application that allows you to run a
-[Waku](https://specs.vac.dev/waku/waku.html) enabled node.
-
-The Waku specification is still in draft and thus this implementation will
-change accordingly.
-
-Additionally the original Whisper (EIP-627) protocol can also be enabled as can
-an experimental Whisper - Waku bridging option.
-
-# How to Build & Run
-
-## Prerequisites
-
-* GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.
-* PCRE
-
-More information on the installation of these can be found [here](https://github.com/status-im/nimbus#prerequisites).
-
-## Build & Run
-
-```bash
-# The first `make` invocation will update all Git submodules.
-# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
-make wakunode
-
-# See available command line options
-./build/wakunode --help
-
-# Connect the client directly with the Status test fleet
-./build/wakunode --log-level:debug --discovery:off --fleet:test --log-metrics
-```
-
-# Using Metrics
-
-Metrics are available for valid envelopes and dropped envelopes.
-
-To compile in an HTTP endpoint for accessing the metrics we need to provide the
-`insecure` flag:
-```bash
-make NIMFLAGS="-d:insecure" wakunode
-./build/wakunode --metrics-server
-```
-
-Ensure your Prometheus config `prometheus.yml` contains the targets you care about, e.g.:
-
-```
-scrape_configs:
-  - job_name: "waku"
-    static_configs:
-      - targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']
-```
-
-For visualisation, similar steps can be used as is written down for Nimbus
-[here](https://github.com/status-im/nimbus#metric-visualisation).
-
-There is a similar example dashboard that includes visualisation of the
-envelopes available at `waku/examples/waku-grafana-dashboard.json`.
-
-# Testing Waku Protocol
-One can set up several nodes, get them connected and then instruct them via the
-JSON-RPC interface. This can be done via e.g. web3.js, nim-web3 (needs to be
-updated) or simply curl your way out.
-
-The JSON-RPC interface is currently the same as the one of Whisper. The only
-difference is the addition of broadcasting the topics interest when a filter
-with a certain set of topics is subcribed.
-
-Example of a quick simulation using this approach:
-```bash
-# Build wakunode + quicksim
-make NIMFLAGS="-d:insecure" wakusim
-
-# Start the simulation nodes, this currently requires multitail to be installed
-./build/start_network --topology:FullMesh --amount:6 --test-node-peers:2
-# In another shell run
-./build/quicksim
-```
-
-The `start_network` tool will also provide a `prometheus.yml` with targets
-set to all simulation nodes that are started. This way you can easily start
-prometheus with this config, e.g.:
-
-```bash
-cd waku/metrics/prometheus
-prometheus
-```
-
-A Grafana dashboard containing the example dashboard for each simulation node
-is also generated and can be imported in case you have Grafana running.
-This dashboard can be found at `./waku/metrics/waku-sim-all-nodes-grafana-dashboard.json`
-
-# Spec support
-
-*This section last updated April 21, 2020*
-
-This client of Waku is spec compliant with [Waku spec v1.0](https://specs.vac.dev/waku/waku.html).
-
-It doesn't yet implement the following recommended features:
-- No support for rate limiting
-- No support for DNS discovery to find Waku nodes
-- It doesn't disconnect a peer if it receives a message before a Status message
-- No support for negotiation with peer supporting multiple versions via Devp2p capabilities in `Hello` packet
-
-Additionally it makes the following choices:
-- It doesn't send message confirmations
-- It has partial support for accounting:
-  - Accounting of total resource usage and total circulated envelopes is done through metrics But no accounting is done for individual peers.

waku/node/v1/metrics/prometheus/prometheus.yml

@@ -1,24 +0,0 @@
-  global:
-    scrape_interval: 1s
-
-  scrape_configs:
-    - job_name: "wakusim"
-      static_configs:
-      - targets: ['127.0.0.1:8010']
-        labels:
-          node: '0'
-      - targets: ['127.0.0.1:8011']
-        labels:
-          node: '1'
-      - targets: ['127.0.0.1:8012']
-        labels:
-          node: '2'
-      - targets: ['127.0.0.1:8013']
-        labels:
-          node: '3'
-      - targets: ['127.0.0.1:8008']
-        labels:
-          node: '4'
-      - targets: ['127.0.0.1:8009']
-        labels:
-          node: '5'

waku/node/v1/start_network.nim

@@ -5,7 +5,7 @@ import
 const
   defaults ="--log-level:DEBUG --log-metrics --metrics-server --rpc"
   wakuNodeBin = "build" / "wakunode"
-  metricsDir = "waku" / "metrics"
+  metricsDir = "metrics"
   portOffset = 2
 
 type
@@ -189,7 +189,7 @@ when isMainModule:
 
   generatePrometheusConfig(nodes, metricsDir / "prometheus" / "prometheus.yml")
   proccessGrafanaDashboard(nodes.len,
-    "waku" / "examples" / "waku-grafana-dashboard.json",
+    metricsDir / "waku-grafana-dashboard.json",
     metricsDir / "waku-sim-all-nodes-grafana-dashboard.json")
 
   let errorCode = execCmd(commandStr)

waku/protocol/README.md

@@ -1,10 +1,7 @@
 # Waku Protocol
 
-TODO.
+This folder contains supported versions of the Waku protocol.
 
-This folder will contain/link to https://github.com/status-im/nim-eth/tree/master/eth/p2p/rlpx_protocols as well as the libp2p version>
+Currently,
-
+- v1 contains the implementation according to Waku specification v1.
-Briefly on versions:
+- v2 is experimental development with Waku build on top of libp2p.
-
-- Waku v0 and v1 are here https://github.com/vacp2p/specs/tree/master/specs/waku
-- Waku v2 will be build on top of libp2p