mirror of https://github.com/waku-org/nwaku.git
Folder restructure (#274)
* Change folder structure to {v1,v2,common}/...
Addresses https://github.com/status-im/nim-waku/issues/261
* Update waku.nimble paths
* Flatten paths
* Fix import paths
* Pull out utils folder for nat
* Pull out waku_types to top level for v2
* Fix test import paths
* Remove old READMEs and replace with one liner
* Update README and split v1 and v2
* Skeleton READMEs
* Update README.md
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
* Update README.md
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
This commit is contained in:
Oskar Thorén
GitHub
parent
d46f357495
commit
9518322198
161
README.md
161
README.md
|
|
@ -1,165 +1,56 @@
|
|||
# nim-waku
|
||||
## Waku v1
|
||||
### Introduction
|
||||
The nim-waku repository holds a Nim implementation of the [Waku v1 protocol](https://specs.vac.dev/waku/waku.html) and a cli application `wakunode` that allows you to run a Waku enabled node from command line.
|
||||
|
||||
The Waku v1 specification is still in draft and thus this implementation will
|
||||
change accordingly. For supported specification details see [here](#spec-support).
|
||||
## Introduction
|
||||
|
||||
Additionally the original Whisper (EIP-627) protocol can also be enabled as can
|
||||
an experimental Whisper - Waku bridging option.
|
||||
The nim-waku repository implements Waku v1 and v2, and provides tools related to it.
|
||||
|
||||
The underlying transport protocol is [rlpx + devp2p](https://github.com/ethereum/devp2p/blob/master/rlpx.md) and the [nim-eth](https://github.com/status-im/nim-eth) implementation is used.
|
||||
- A Nim implementation of the [Waku v1 protocol](https://specs.vac.dev/waku/waku.html).
|
||||
- A Nim implementation of the [Waku v2 protocol](https://specs.vac.dev/specs/waku/v2/waku-v2.html).
|
||||
- CLI applications `wakunode` and `wakunode2` that allows you to run a Waku v1 or v2 node.
|
||||
- Examples of Waku v1 and v2 usage.
|
||||
- Various tests of above.
|
||||
|
||||
This repository is also a place for experimenting with possible future versions
|
||||
of Waku such as replacing the transport protocol with libp2p, see also [Waku v2 protocol](#waku-v2)
|
||||
For more details on Waku v1 and v2, see their respective home folders:
|
||||
|
||||
### How to Build & Run
|
||||
- [Waku v1](waku/v1/README.md)
|
||||
- [Waku v2](waku/v2/README.md)
|
||||
|
||||
#### Prerequisites
|
||||
## How to Build & Run
|
||||
|
||||
These instructions are generic and apply to both Waku v1 and v2. For more
|
||||
detailed instructions, see Waku v1 and v2 home above.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
* GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.
|
||||
* PCRE
|
||||
|
||||
More information on the installation of these can be found [here](https://github.com/status-im/nimbus#prerequisites).
|
||||
|
||||
#### Wakunode
|
||||
### Wakunode
|
||||
|
||||
```bash
|
||||
# The first `make` invocation will update all Git submodules.
|
||||
# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
|
||||
make wakunode1
|
||||
make wakunode1 wakunode2
|
||||
|
||||
# See available command line options
|
||||
./build/wakunode --help
|
||||
./build/wakunode2 --help
|
||||
|
||||
# Connect the client directly with the Status test fleet
|
||||
./build/wakunode --log-level:debug --discovery:off --fleet:test --log-metrics
|
||||
# TODO Equivalent for v2
|
||||
```
|
||||
|
||||
#### Waku v1 Protocol Test Suite
|
||||
### Waku Protocol Test Suite
|
||||
|
||||
```bash
|
||||
# Run all the Waku v1 tests
|
||||
make test1
|
||||
# Run all the Waku v1 and v2 tests
|
||||
make test
|
||||
```
|
||||
|
||||
You can also run a specific test (and alter compile options as you want):
|
||||
```bash
|
||||
# Get a shell with the right environment variables set
|
||||
./env.sh bash
|
||||
# Run a specific test
|
||||
nim c -r ./tests/v1/test_waku_connect.nim
|
||||
```
|
||||
### Examples
|
||||
|
||||
#### Waku v1 Protocol Example
|
||||
There is a more basic example, more limited in features and configuration than
|
||||
the `wakunode`, located in `examples/v1/example.nim`.
|
||||
|
||||
More information on how to run this example can be found it its
|
||||
[readme](examples/v1/README.md).
|
||||
|
||||
#### Waku Quick Simulation
|
||||
One can set up several nodes, get them connected and then instruct them via the
|
||||
JSON-RPC interface. This can be done via e.g. web3.js, nim-web3 (needs to be
|
||||
updated) or simply curl your way out.
|
||||
|
||||
The JSON-RPC interface is currently the same as the one of Whisper. The only
|
||||
difference is the addition of broadcasting the topics interest when a filter
|
||||
with a certain set of topics is subcribed.
|
||||
|
||||
The quick simulation uses this approach, `start_network` launches a set of
|
||||
`wakunode`s, and `quicksim` instructs the nodes through RPC calls.
|
||||
|
||||
Example of how to build and run:
|
||||
```bash
|
||||
# Build wakunode + quicksim with metrics enabled
|
||||
make NIMFLAGS="-d:insecure" sim1
|
||||
|
||||
# Start the simulation nodes, this currently requires multitail to be installed
|
||||
./build/start_network --topology:FullMesh --amount:6 --test-node-peers:2
|
||||
# In another shell run
|
||||
./build/quicksim
|
||||
```
|
||||
|
||||
The `start_network` tool will also provide a `prometheus.yml` with targets
|
||||
set to all simulation nodes that are started. This way you can easily start
|
||||
prometheus with this config, e.g.:
|
||||
|
||||
```bash
|
||||
cd ./metrics/prometheus
|
||||
prometheus
|
||||
```
|
||||
|
||||
A Grafana dashboard containing the example dashboard for each simulation node
|
||||
is also generated and can be imported in case you have Grafana running.
|
||||
This dashboard can be found at `./metrics/waku-sim-all-nodes-grafana-dashboard.json`
|
||||
|
||||
To read more details about metrics, see [next](#using-metrics) section.
|
||||
|
||||
### Using Metrics
|
||||
|
||||
Metrics are available for valid envelopes and dropped envelopes.
|
||||
|
||||
To compile in an HTTP endpoint for accessing the metrics we need to provide the
|
||||
`insecure` flag:
|
||||
```bash
|
||||
make NIMFLAGS="-d:insecure" wakunode1
|
||||
./build/wakunode --metrics-server
|
||||
```
|
||||
|
||||
Ensure your Prometheus config `prometheus.yml` contains the targets you care about, e.g.:
|
||||
|
||||
```
|
||||
scrape_configs:
|
||||
- job_name: "waku"
|
||||
static_configs:
|
||||
- targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']
|
||||
```
|
||||
|
||||
For visualisation, similar steps can be used as is written down for Nimbus
|
||||
[here](https://github.com/status-im/nimbus#metric-visualisation).
|
||||
|
||||
There is a similar example dashboard that includes visualisation of the
|
||||
envelopes available at `metrics/waku-grafana-dashboard.json`.
|
||||
|
||||
### Spec support
|
||||
|
||||
*This section last updated April 21, 2020*
|
||||
|
||||
This client of Waku is spec compliant with [Waku spec v1.0.0](https://specs.vac.dev/waku/waku.html).
|
||||
|
||||
It doesn't yet implement the following recommended features:
|
||||
- No support for rate limiting
|
||||
- No support for DNS discovery to find Waku nodes
|
||||
- It doesn't disconnect a peer if it receives a message before a Status message
|
||||
- No support for negotiation with peer supporting multiple versions via Devp2p capabilities in `Hello` packet
|
||||
|
||||
Additionally it makes the following choices:
|
||||
- It doesn't send message confirmations
|
||||
- It has partial support for accounting:
|
||||
- Accounting of total resource usage and total circulated envelopes is done through metrics But no accounting is done for individual peers.
|
||||
|
||||
## Waku v2
|
||||
|
||||
Waku v2 is under active development but is currently in an early alpha state.
|
||||
See `waku/node/v2/` and `waku/protocol/v2/` directory for more details on the current
|
||||
state.
|
||||
|
||||
Here's a post outlining the [current plan for Waku
|
||||
v2](https://vac.dev/waku-v2-plan), and here's the current roadmap and progress
|
||||
https://github.com/vacp2p/research/issues/40
|
||||
|
||||
## Docker Image
|
||||
You can create a Docker image using:
|
||||
```bash
|
||||
make docker-image
|
||||
docker run --rm -it statusteam/nim-waku:latest --help
|
||||
```
|
||||
|
||||
Default, the target will be a docker image with `wakunode`, which is the Waku v1 node.
|
||||
You can change this to `wakunode2`, the Waku v2 node like this:
|
||||
```bash
|
||||
make docker-image MAKE_TARGET=wakunode2
|
||||
docker run --rm -it statusteam/nim-waku:latest --help
|
||||
```
|
||||
Examples can be found in the examples folder. For Waku v2, there is a fully
|
||||
featured chat example.
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
import
|
||||
confutils, chronicles, chronos, stew/byteutils, stew/shims/net as stewNet,
|
||||
eth/[keys, p2p],
|
||||
../../waku/protocol/v1/waku_protocol,
|
||||
../../waku/node/v1/waku_helpers,
|
||||
../../waku/node/common,
|
||||
../../waku/v1/protocol/waku_protocol,
|
||||
../../waku/v1/node/waku_helpers,
|
||||
../../waku/common/utils/nat,
|
||||
./config_example
|
||||
|
||||
## This is a simple Waku v1 example to show the Waku v1 API usage.
|
||||
|
|
|
|||
|
|
@ -8,8 +8,10 @@ import
|
|||
libp2p/crypto/[crypto,secp],
|
||||
eth/keys,
|
||||
json_rpc/[rpcclient, rpcserver],
|
||||
../../waku/node/v2/[config, wakunode2, waku_types],
|
||||
../../waku/node/common
|
||||
../../waku/v2/node/[config, wakunode2],
|
||||
../../waku/common/utils/nat,
|
||||
../../waku/v2/waku_types
|
||||
|
||||
|
||||
type
|
||||
Topic* = waku_types.Topic
|
||||
|
|
|
|||
|
|
@ -19,9 +19,10 @@ import libp2p/[switch, # manage transports, a single entry poi
|
|||
protocols/secure/secio, # define the protocol of secure input / output, allows encrypted communication that uses public keys to validate signed messages instead of a certificate authority like in TLS
|
||||
muxers/muxer, # define an interface for stream multiplexing, allowing peers to offer many protocols over a single connection
|
||||
muxers/mplex/mplex] # define some contants and message types for stream multiplexing
|
||||
import ../../waku/node/v2/[config, wakunode2, waku_types, waku_payload],
|
||||
../../waku/protocol/v2/[waku_relay, waku_store, waku_filter],
|
||||
../../waku/node/common
|
||||
import ../../waku/v2/node/[config, wakunode2, waku_payload],
|
||||
../../waku/v2/protocol/[waku_relay, waku_store, waku_filter],
|
||||
../../waku/common/utils/nat,
|
||||
../../waku/v2/waku_types
|
||||
|
||||
const Help = """
|
||||
Commands: /[?|help|connect|disconnect|exit]
|
||||
|
|
|
|||
|
|
@ -4,13 +4,13 @@ import
|
|||
std/[unittest, options, os, strutils],
|
||||
stew/byteutils, json_rpc/[rpcserver, rpcclient],
|
||||
eth/common as eth_common, eth/[rlp, keys, p2p],
|
||||
../../waku/protocol/v1/waku_protocol,
|
||||
../../waku/node/v1/rpc/[hexstrings, rpc_types, waku, key_storage]
|
||||
../../waku/v1/protocol/waku_protocol,
|
||||
../../waku/v1/node/rpc/[hexstrings, rpc_types, waku, key_storage]
|
||||
|
||||
template sourceDir*: string = currentSourcePath.rsplit(DirSep, 1)[0]
|
||||
## Generate client convenience marshalling wrappers from forward declarations
|
||||
## For testing, ethcallsigs needs to be kept in sync with ../waku/node/v1/rpc/waku
|
||||
const sigPath = sourceDir / ParDir / ParDir / "waku" / "node" / "v1" / "rpc" / "wakucallsigs.nim"
|
||||
const sigPath = sourceDir / ParDir / ParDir / "waku" / "v1" / "node" / "rpc" / "wakucallsigs.nim"
|
||||
createRpcSigs(RpcSocketClient, sigPath)
|
||||
|
||||
proc setupNode(capabilities: varargs[ProtocolInfo, `protocolInfo`],
|
||||
|
|
|
|||
|
|
@ -12,8 +12,8 @@ import
|
|||
std/[sequtils, unittest, tables],
|
||||
chronos, eth/p2p, eth/p2p/peer_pool,
|
||||
eth/p2p/rlpx_protocols/whisper_protocol as whisper,
|
||||
../../waku/protocol/v1/waku_protocol as waku,
|
||||
../../waku/protocol/v1/waku_bridge,
|
||||
../../waku/v1/protocol/waku_protocol as waku,
|
||||
../../waku/v1/protocol/waku_bridge,
|
||||
../test_helpers
|
||||
|
||||
let safeTTL = 5'u32
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@
|
|||
|
||||
import
|
||||
std/[sequtils, options, unittest, times],
|
||||
../../waku/protocol/v1/waku_protocol
|
||||
../../waku/v1/protocol/waku_protocol
|
||||
|
||||
suite "Waku envelope validation":
|
||||
test "should validate and allow envelope according to config":
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@
|
|||
import
|
||||
std/[sequtils, tables, unittest],
|
||||
chronos, eth/[keys, p2p], eth/p2p/peer_pool,
|
||||
../../waku/protocol/v1/waku_protocol,
|
||||
../../waku/v1/protocol/waku_protocol,
|
||||
../test_helpers
|
||||
|
||||
const
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
import
|
||||
std/[unittest, tables, sequtils, times],
|
||||
chronos, eth/[p2p, async_utils], eth/p2p/peer_pool,
|
||||
../../waku/protocol/v1/[waku_protocol, waku_mail],
|
||||
../../waku/v1/protocol/[waku_protocol, waku_mail],
|
||||
../test_helpers
|
||||
|
||||
const
|
||||
|
|
|
|||
|
|
@ -2,8 +2,9 @@
|
|||
import
|
||||
std/[unittest, options, tables, sets],
|
||||
chronos, chronicles,
|
||||
../../waku/node/v2/[waku_types, message_store],
|
||||
../test_helpers, ./utils
|
||||
../../waku/v2/node/message_store,
|
||||
../test_helpers, ./utils,
|
||||
../../waku/v2/waku_types
|
||||
|
||||
suite "Message Store":
|
||||
test "set and get works":
|
||||
|
|
|
|||
|
|
@ -5,13 +5,15 @@ import
|
|||
stew/shims/net as stewNet,
|
||||
json_rpc/[rpcserver, rpcclient],
|
||||
libp2p/crypto/crypto,
|
||||
../../waku/node/v2/[waku_types, wakunode2],
|
||||
../../waku/node/v2/rpc/wakurpc,
|
||||
../../waku/protocol/v2/waku_relay,
|
||||
../../waku/v2/node/wakunode2,
|
||||
../../waku/v2/node/rpc/wakurpc,
|
||||
../../waku/v2/protocol/waku_relay,
|
||||
../../waku/v2/waku_types,
|
||||
../test_helpers
|
||||
|
||||
|
||||
template sourceDir*: string = currentSourcePath.rsplit(DirSep, 1)[0]
|
||||
const sigPath = sourceDir / ParDir / ParDir / "waku" / "node" / "v2" / "rpc" / "wakucallsigs.nim"
|
||||
const sigPath = sourceDir / ParDir / ParDir / "waku" / "v2" / "node" / "rpc" / "wakucallsigs.nim"
|
||||
createRpcSigs(RpcHttpClient, sigPath)
|
||||
|
||||
suite "Waku v2 Remote Procedure Calls":
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ import utils,
|
|||
libp2p/stream/[bufferstream, connection],
|
||||
libp2p/crypto/crypto,
|
||||
libp2p/protocols/pubsub/floodsub
|
||||
import ../../waku/protocol/v2/waku_relay
|
||||
import ../../waku/v2/protocol/waku_relay
|
||||
|
||||
import ../test_helpers
|
||||
|
||||
|
|
|
|||
|
|
@ -10,8 +10,8 @@ import
|
|||
libp2p/multistream,
|
||||
libp2p/transports/transport,
|
||||
libp2p/transports/tcptransport,
|
||||
../../waku/protocol/v2/[waku_relay, waku_filter, message_notifier],
|
||||
../../waku/node/v2/waku_types,
|
||||
../../waku/v2/protocol/[waku_relay, waku_filter, message_notifier],
|
||||
../../waku/v2/waku_types,
|
||||
../test_helpers, ./utils
|
||||
|
||||
procSuite "Waku Filter":
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
import
|
||||
std/[unittest,algorithm,options],
|
||||
nimcrypto/sha2,
|
||||
../../waku/node/v2/waku_types,
|
||||
../../waku/protocol/v2/waku_store,
|
||||
../../waku/v2/waku_types,
|
||||
../../waku/v2/protocol/waku_store,
|
||||
../test_helpers
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
import
|
||||
std/unittest,
|
||||
../../waku/node/v2/waku_payload,
|
||||
../../waku/v2/node/waku_payload,
|
||||
../test_helpers
|
||||
|
||||
procSuite "Waku Payload":
|
||||
|
|
|
|||
|
|
@ -11,9 +11,10 @@ import
|
|||
libp2p/multistream,
|
||||
libp2p/transports/transport,
|
||||
libp2p/transports/tcptransport,
|
||||
../../waku/protocol/v2/[waku_store, message_notifier],
|
||||
../../waku/node/v2/[waku_types, message_store],
|
||||
../test_helpers, ./utils
|
||||
../../waku/v2/protocol/[waku_store, message_notifier],
|
||||
../../waku/v2/node/message_store,
|
||||
../test_helpers, ./utils,
|
||||
../../waku/v2/waku_types
|
||||
|
||||
|
||||
procSuite "Waku Store":
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
import
|
||||
std/[unittest, options, tables, sets],
|
||||
chronos, chronicles,
|
||||
../../waku/protocol/v2/waku_swap,
|
||||
../../waku/node/v2/waku_types,
|
||||
../../waku/v2/protocol/waku_swap,
|
||||
../../waku/v2/waku_types,
|
||||
../test_helpers, ./utils
|
||||
|
||||
procSuite "Waku SWAP Accounting":
|
||||
|
|
|
|||
|
|
@ -7,9 +7,10 @@ import
|
|||
libp2p/crypto/secp,
|
||||
libp2p/switch,
|
||||
eth/keys,
|
||||
../../waku/protocol/v2/[waku_relay, waku_store, waku_filter, message_notifier],
|
||||
../../waku/node/v2/[wakunode2, waku_types],
|
||||
../test_helpers
|
||||
../../waku/v2/protocol/[waku_relay, waku_store, waku_filter, message_notifier],
|
||||
../../waku/v2/node/wakunode2,
|
||||
../test_helpers,
|
||||
../../waku/v2/waku_types
|
||||
|
||||
procSuite "WakuNode":
|
||||
let rng = keys.newRng()
|
||||
|
|
|
|||
|
|
@ -10,8 +10,8 @@ import libp2p/[standard_setup,
|
|||
protocols/pubsub/floodsub,
|
||||
protocols/pubsub/gossipsub,
|
||||
protocols/secure/secure]
|
||||
import ../../waku/protocol/v2/waku_relay,
|
||||
../../waku/node/v2/waku_types
|
||||
import ../../waku/v2/protocol/waku_relay,
|
||||
../../waku/v2/waku_types
|
||||
|
||||
export standard_setup
|
||||
|
||||
|
|
|
|||
24
waku.nimble
24
waku.nimble
|
|
@ -42,11 +42,11 @@ proc test(name: string, lang = "c") =
|
|||
|
||||
### Waku v1 tasks
|
||||
task wakunode1, "Build Waku v1 cli node":
|
||||
buildBinary "wakunode", "waku/node/v1/", "-d:chronicles_log_level=TRACE"
|
||||
buildBinary "wakunode", "waku/v1/node/", "-d:chronicles_log_level=TRACE"
|
||||
|
||||
task sim1, "Build Waku v1 simulation tools":
|
||||
buildBinary "quicksim", "waku/node/v1/", "-d:chronicles_log_level=INFO"
|
||||
buildBinary "start_network", "waku/node/v1/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "quicksim", "waku/v1/node/", "-d:chronicles_log_level=INFO"
|
||||
buildBinary "start_network", "waku/v1/node/", "-d:chronicles_log_level=DEBUG"
|
||||
|
||||
task example1, "Build Waku v1 example":
|
||||
buildBinary "example", "examples/v1/", "-d:chronicles_log_level=DEBUG"
|
||||
|
|
@ -56,11 +56,11 @@ task test1, "Build & run Waku v1 tests":
|
|||
|
||||
### Waku v2 tasks
|
||||
task wakunode2, "Build Waku v2 (experimental) cli node":
|
||||
buildBinary "wakunode2", "waku/node/v2/", "-d:chronicles_log_level=TRACE"
|
||||
buildBinary "wakunode2", "waku/v2/node/", "-d:chronicles_log_level=TRACE"
|
||||
|
||||
task sim2, "Build Waku v2 simulation tools":
|
||||
buildBinary "quicksim2", "waku/node/v2/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "start_network2", "waku/node/v2/", "-d:chronicles_log_level=TRACE"
|
||||
buildBinary "quicksim2", "waku/v2/node/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "start_network2", "waku/v2/node/", "-d:chronicles_log_level=TRACE"
|
||||
|
||||
task example2, "Build Waku v2 example":
|
||||
let name = "basic2"
|
||||
|
|
@ -70,11 +70,11 @@ task test2, "Build & run Waku v2 tests":
|
|||
test "all_tests_v2"
|
||||
|
||||
task scripts2, "Build Waku v2 scripts":
|
||||
buildBinary "rpc_publish", "waku/node/v2/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_subscribe", "waku/node/v2/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_subscribe_filter", "waku/node/v2/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_query", "waku/node/v2/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_info", "waku/node/v2/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_publish", "waku/v2/node/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_subscribe", "waku/v2/node/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_subscribe_filter", "waku/v2/node/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_query", "waku/v2/node/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "rpc_info", "waku/v2/node/rpc/", "-d:chronicles_log_level=DEBUG"
|
||||
|
||||
task chat2, "Build example Waku v2 chat usage":
|
||||
let name = "chat2"
|
||||
|
|
@ -84,4 +84,4 @@ task chat2, "Build example Waku v2 chat usage":
|
|||
buildBinary name, "examples/v2/", "-d:chronicles_log_level=DEBUG"
|
||||
|
||||
task bridge, "Build Waku v1 - v2 bridge":
|
||||
buildBinary "wakubridge", "waku/node/", "-d:chronicles_log_level=DEBUG"
|
||||
buildBinary "wakubridge", "waku/common/", "-d:chronicles_log_level=DEBUG"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,7 @@
|
|||
# Common
|
||||
|
||||
This folder contains (a) modules that use both Waku v1 and Waku v2. and (b) utilities that are useful for both Waku v1 and v2.
|
||||
|
||||
Examples include:
|
||||
- Bridge between v1 and v2
|
||||
- NAT traversal
|
||||
|
|
@ -5,12 +5,14 @@ import
|
|||
# Waku v1 imports
|
||||
eth/[keys, p2p], eth/common/utils,
|
||||
eth/p2p/[enode, whispernodes],
|
||||
../protocol/v1/waku_protocol, ./common,
|
||||
./v1/rpc/[waku, wakusim, key_storage], ./v1/waku_helpers,
|
||||
../v1/protocol/waku_protocol,
|
||||
./utils/nat,
|
||||
../v1/node/rpc/[waku, wakusim, key_storage],
|
||||
../v1/node/waku_helpers,
|
||||
# Waku v2 imports
|
||||
libp2p/crypto/crypto,
|
||||
./v2/wakunode2,
|
||||
./v2/rpc/wakurpc,
|
||||
../v2/node/wakunode2,
|
||||
../v2/node/rpc/wakurpc,
|
||||
# Common cli config
|
||||
./config_bridge
|
||||
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
# Waku Node
|
||||
|
||||
This folder contains Waku node implementations and simulations.
|
||||
|
||||
Currently,
|
||||
- v1 contains the implementation according to Waku specification v1.
|
||||
- v2 is experimental development with Waku build on top of libp2p.
|
||||
|
||||
## Run simulation
|
||||
|
||||
To get metrics server, etc.
|
||||
|
||||
`make NIMFLAGS="-d:insecure" wakusim2`
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
# Waku node v2
|
||||
|
||||
Status: pre-alpha
|
||||
|
||||
Right now a hacky PoC for running on libp2p.
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
# Waku Protocol
|
||||
|
||||
This folder contains supported versions of the Waku protocol.
|
||||
|
||||
Currently,
|
||||
- v1 contains the implementation according to Waku specification v1.
|
||||
- v2 is experimental development with Waku build on top of libp2p.
|
||||
|
|
@ -0,0 +1,156 @@
|
|||
# Waku v1
|
||||
|
||||
This folder contains code related to Waku v1, both as a node and as a protocol.
|
||||
|
||||
## Introduction
|
||||
|
||||
This is a Nim implementation of the Nim implementation of the [Waku v1 protocol](https://specs.vac.dev/waku/waku.html) and a cli application `wakunode` that allows you to run a Waku enabled node from command line.
|
||||
|
||||
For supported specification details see [here](#spec-support).
|
||||
|
||||
Additionally the original Whisper (EIP-627) protocol can also be enabled as can
|
||||
an experimental Whisper - Waku bridging option.
|
||||
|
||||
The underlying transport protocol is [rlpx + devp2p](https://github.com/ethereum/devp2p/blob/master/rlpx.md) and the [nim-eth](https://github.com/status-im/nim-eth) implementation is used.
|
||||
|
||||
## How to Build & Run
|
||||
|
||||
All of the below commands should be executed at the root level, i.e. `cd ../..`.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
* GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.
|
||||
* PCRE
|
||||
|
||||
More information on the installation of these can be found [here](https://github.com/status-im/nimbus#prerequisites).
|
||||
|
||||
### Wakunode
|
||||
|
||||
```bash
|
||||
# The first `make` invocation will update all Git submodules.
|
||||
# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
|
||||
make wakunode1
|
||||
|
||||
# See available command line options
|
||||
./build/wakunode --help
|
||||
|
||||
# Connect the client directly with the Status test fleet
|
||||
./build/wakunode --log-level:debug --discovery:off --fleet:test --log-metrics
|
||||
```
|
||||
|
||||
### Waku v1 Protocol Test Suite
|
||||
|
||||
```bash
|
||||
# Run all the Waku v1 tests
|
||||
make test1
|
||||
```
|
||||
|
||||
You can also run a specific test (and alter compile options as you want):
|
||||
|
||||
```bash
|
||||
# Get a shell with the right environment variables set
|
||||
./env.sh bash
|
||||
# Run a specific test
|
||||
nim c -r ./tests/v1/test_waku_connect.nim
|
||||
```
|
||||
|
||||
### Waku v1 Protocol Example
|
||||
|
||||
There is a more basic example, more limited in features and configuration than
|
||||
the `wakunode`, located in `examples/v1/example.nim`.
|
||||
|
||||
More information on how to run this example can be found it its
|
||||
[readme](../../examples/v1/README.md).
|
||||
|
||||
### Waku Quick Simulation
|
||||
|
||||
One can set up several nodes, get them connected and then instruct them via the
|
||||
JSON-RPC interface. This can be done via e.g. web3.js, nim-web3 (needs to be
|
||||
updated) or simply curl your way out.
|
||||
|
||||
The JSON-RPC interface is currently the same as the one of Whisper. The only
|
||||
difference is the addition of broadcasting the topics interest when a filter
|
||||
with a certain set of topics is subcribed.
|
||||
|
||||
The quick simulation uses this approach, `start_network` launches a set of
|
||||
`wakunode`s, and `quicksim` instructs the nodes through RPC calls.
|
||||
|
||||
Example of how to build and run:
|
||||
```bash
|
||||
# Build wakunode + quicksim with metrics enabled
|
||||
make NIMFLAGS="-d:insecure" sim1
|
||||
|
||||
# Start the simulation nodes, this currently requires multitail to be installed
|
||||
./build/start_network --topology:FullMesh --amount:6 --test-node-peers:2
|
||||
# In another shell run
|
||||
./build/quicksim
|
||||
```
|
||||
|
||||
The `start_network` tool will also provide a `prometheus.yml` with targets
|
||||
set to all simulation nodes that are started. This way you can easily start
|
||||
prometheus with this config, e.g.:
|
||||
|
||||
```bash
|
||||
cd ./metrics/prometheus
|
||||
prometheus
|
||||
```
|
||||
|
||||
A Grafana dashboard containing the example dashboard for each simulation node
|
||||
is also generated and can be imported in case you have Grafana running.
|
||||
This dashboard can be found at `./metrics/waku-sim-all-nodes-grafana-dashboard.json`
|
||||
|
||||
To read more details about metrics, see [next](#using-metrics) section.
|
||||
|
||||
## Using Metrics
|
||||
|
||||
Metrics are available for valid envelopes and dropped envelopes.
|
||||
|
||||
To compile in an HTTP endpoint for accessing the metrics we need to provide the
|
||||
`insecure` flag:
|
||||
```bash
|
||||
make NIMFLAGS="-d:insecure" wakunode1
|
||||
./build/wakunode --metrics-server
|
||||
```
|
||||
|
||||
Ensure your Prometheus config `prometheus.yml` contains the targets you care about, e.g.:
|
||||
|
||||
```
|
||||
scrape_configs:
|
||||
- job_name: "waku"
|
||||
static_configs:
|
||||
- targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']
|
||||
```
|
||||
|
||||
For visualisation, similar steps can be used as is written down for Nimbus
|
||||
[here](https://github.com/status-im/nimbus#metric-visualisation).
|
||||
|
||||
There is a similar example dashboard that includes visualisation of the
|
||||
envelopes available at `metrics/waku-grafana-dashboard.json`.
|
||||
|
||||
## Spec support
|
||||
|
||||
*This section last updated April 21, 2020*
|
||||
|
||||
This client of Waku is spec compliant with [Waku spec v1.0.0](https://specs.vac.dev/waku/waku.html).
|
||||
|
||||
It doesn't yet implement the following recommended features:
|
||||
- No support for rate limiting
|
||||
- No support for DNS discovery to find Waku nodes
|
||||
- It doesn't disconnect a peer if it receives a message before a Status message
|
||||
- No support for negotiation with peer supporting multiple versions via Devp2p capabilities in `Hello` packet
|
||||
|
||||
Additionally it makes the following choices:
|
||||
- It doesn't send message confirmations
|
||||
- It has partial support for accounting:
|
||||
- Accounting of total resource usage and total circulated envelopes is done through metrics But no accounting is done for individual peers.
|
||||
|
||||
## Docker Image
|
||||
|
||||
You can create a Docker image using:
|
||||
|
||||
```bash
|
||||
make docker-image
|
||||
docker run --rm -it statusteam/nim-waku:latest --help
|
||||
```
|
||||
|
||||
The target will be a docker image with `wakunode`, which is the Waku v1 node.
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Waku Node v1
|
||||
|
||||
This folder contains code related to running a `wakunode` process. The main entrypoint is the `wakunode` file.
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
import
|
||||
os, strformat, chronicles, json_rpc/[rpcclient, rpcserver], nimcrypto/sysrand,
|
||||
eth/common as eth_common, eth/keys,
|
||||
../../protocol/v1/waku_protocol, ./rpc/[hexstrings, rpc_types],
|
||||
../protocol/waku_protocol, ./rpc/[hexstrings, rpc_types],
|
||||
options as what # TODO: Huh? Redefinition?
|
||||
|
||||
from os import DirSep
|
||||
|
|
@ -26,7 +26,7 @@
|
|||
|
||||
import
|
||||
stint, stew/byteutils, eth/[keys, rlp], eth/common/eth_types,
|
||||
../../../protocol/v1/waku_protocol
|
||||
../../protocol/waku_protocol
|
||||
|
||||
type
|
||||
HexDataStr* = distinct string
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
import
|
||||
hexstrings, options, eth/[keys, rlp],
|
||||
../../../protocol/v1/waku_protocol
|
||||
../../protocol/waku_protocol
|
||||
|
||||
#[
|
||||
Notes:
|
||||
|
|
@ -3,7 +3,7 @@ import
|
|||
eth/[common, rlp, keys, p2p],
|
||||
nimcrypto/[sysrand, hmac, sha2, pbkdf2],
|
||||
rpc_types, hexstrings, key_storage,
|
||||
../../../protocol/v1/waku_protocol
|
||||
../../protocol/waku_protocol
|
||||
|
||||
from stew/byteutils import hexToSeqByte, hexToByteArray
|
||||
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
import
|
||||
json_rpc/rpcserver, stew/endians2, nimcrypto/sysrand,
|
||||
eth/[p2p, async_utils],
|
||||
../../../protocol/v1/waku_protocol
|
||||
../../protocol/waku_protocol
|
||||
|
||||
proc generateTraffic(node: EthereumNode, amount = 100) {.async.} =
|
||||
let payload = @[byte 0]
|
||||
|
|
@ -4,7 +4,8 @@ import
|
|||
eth/[keys, p2p], eth/common/utils,
|
||||
eth/p2p/[discovery, enode, peer_pool, bootnodes, whispernodes],
|
||||
eth/p2p/rlpx_protocols/whisper_protocol,
|
||||
../../protocol/v1/[waku_protocol, waku_bridge], ../common,
|
||||
../protocol/[waku_protocol, waku_bridge],
|
||||
../../common/utils/nat,
|
||||
./rpc/[waku, wakusim, key_storage], ./waku_helpers, ./config
|
||||
|
||||
const clientId = "Nimbus waku node"
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Waku v1 protocol
|
||||
|
||||
This folder contains implementations of [Waku v1 protocols](https://specs.vac.dev/specs/waku/v1/waku-1.html).
|
||||
|
|
@ -0,0 +1,139 @@
|
|||
# Waku v2
|
||||
|
||||
This folder contains code related to Waku v1, both as a node and as a protocol.
|
||||
|
||||
## Introduction
|
||||
|
||||
This is an implementation in Nim of Waku v2, which is currently in draft/beta stage.
|
||||
|
||||
See [spec](https://specs.vac.dev/specs/waku/v2/waku-v2.html).
|
||||
|
||||
## How to Build & Run
|
||||
|
||||
### Prerequisites
|
||||
|
||||
* GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.
|
||||
* PCRE
|
||||
|
||||
More information on the installation of these can be found [here](https://github.com/status-im/nimbus#prerequisites).
|
||||
|
||||
### Wakunode
|
||||
|
||||
```bash
|
||||
# The first `make` invocation will update all Git submodules.
|
||||
# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
|
||||
make wakunode2
|
||||
|
||||
# See available command line options
|
||||
./build/wakunode2 --help
|
||||
|
||||
# Connect the client directly with the Status test fleet
|
||||
# TODO NYI
|
||||
#./build/wakunode2 --log-level:debug --discovery:off --fleet:test --log-metrics
|
||||
```
|
||||
|
||||
### Waku v2 Protocol Test Suite
|
||||
|
||||
```bash
|
||||
# Run all the Waku v2 tests
|
||||
make test2
|
||||
```
|
||||
|
||||
You can also run a specific test (and alter compile options as you want):
|
||||
```bash
|
||||
# Get a shell with the right environment variables set
|
||||
./env.sh bash
|
||||
# Run a specific test
|
||||
nim c -r ./tests/v2/test_waku_filter.nim
|
||||
```
|
||||
|
||||
### Waku v2 Protocol Example
|
||||
|
||||
There is a more basic example, more limited in features and configuration than
|
||||
the `wakunode1`, located in `examples/v2/basic2.nim`.
|
||||
|
||||
There is also a more full featured example in `examples/v2/chat2.nim`.
|
||||
|
||||
### Waku Quick Simulation
|
||||
|
||||
*NOTE: This section might be slightly out of date as it was written for Waku v1.*
|
||||
|
||||
One can set up several nodes, get them connected and then instruct them via the
|
||||
JSON-RPC interface. This can be done via e.g. web3.js, nim-web3 (needs to be
|
||||
updated) or simply curl your way out.
|
||||
|
||||
The JSON-RPC interface is currently the same as the one of Whisper. The only
|
||||
difference is the addition of broadcasting the topics interest when a filter
|
||||
with a certain set of topics is subcribed.
|
||||
|
||||
The quick simulation uses this approach, `start_network` launches a set of
|
||||
`wakunode`s, and `quicksim` instructs the nodes through RPC calls.
|
||||
|
||||
Example of how to build and run:
|
||||
```bash
|
||||
# Build wakunode + quicksim with metrics enabled
|
||||
make NIMFLAGS="-d:insecure" wakusim2
|
||||
|
||||
# Start the simulation nodes, this currently requires multitail to be installed
|
||||
# TODO Partial support for Waku v2
|
||||
./build/start_network2 --topology:FullMesh --amount:6 --test-node-peers:2
|
||||
# In another shell run
|
||||
./build/quicksim2
|
||||
```
|
||||
|
||||
The `start_network2` tool will also provide a `prometheus.yml` with targets
|
||||
set to all simulation nodes that are started. This way you can easily start
|
||||
prometheus with this config, e.g.:
|
||||
|
||||
```bash
|
||||
cd ./metrics/prometheus
|
||||
prometheus
|
||||
```
|
||||
|
||||
A Grafana dashboard containing the example dashboard for each simulation node
|
||||
is also generated and can be imported in case you have Grafana running.
|
||||
This dashboard can be found at `./metrics/waku-sim-all-nodes-grafana-dashboard.json`
|
||||
|
||||
To read more details about metrics, see [next](#using-metrics) section.
|
||||
|
||||
## Using Metrics
|
||||
|
||||
Metrics are available for valid envelopes and dropped envelopes.
|
||||
|
||||
To compile in an HTTP endpoint for accessing the metrics we need to provide the
|
||||
`insecure` flag:
|
||||
```bash
|
||||
make NIMFLAGS="-d:insecure" wakunode2
|
||||
./build/wakunode2 --metrics-server
|
||||
```
|
||||
|
||||
Ensure your Prometheus config `prometheus.yml` contains the targets you care about, e.g.:
|
||||
|
||||
```
|
||||
scrape_configs:
|
||||
- job_name: "waku"
|
||||
static_configs:
|
||||
- targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']
|
||||
```
|
||||
|
||||
For visualisation, similar steps can be used as is written down for Nimbus
|
||||
[here](https://github.com/status-im/nimbus#metric-visualisation).
|
||||
|
||||
There is a similar example dashboard that includes visualisation of the
|
||||
envelopes available at `metrics/waku-grafana-dashboard.json`.
|
||||
|
||||
## Spec support
|
||||
|
||||
*This section last updated November 16, 2020*
|
||||
|
||||
All Waku v2 specs, except for bridge, are currently in draft.
|
||||
|
||||
## Docker Image
|
||||
|
||||
By default, the target will be a docker image with `wakunode`, which is the Waku v1 node.
|
||||
You can change this to `wakunode2`, the Waku v2 node like this:
|
||||
|
||||
```bash
|
||||
make docker-image MAKE_TARGET=wakunode2
|
||||
docker run --rm -it statusteam/nim-waku:latest --help
|
||||
```
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
# Waku Node v2
|
||||
|
||||
This folder contains code related to running a `wakunode2` process. The main entrypoint is the `wakunode2` file.
|
||||
|
||||
See `../../docs/api/v2/node.md` for more details on the the Nim Node API.
|
||||
|
|
@ -1,13 +1,13 @@
|
|||
import
|
||||
os,
|
||||
sqlite3_abi,
|
||||
waku_types,
|
||||
chronos, chronicles, metrics, stew/results,
|
||||
libp2p/crypto/crypto,
|
||||
libp2p/protocols/protocol,
|
||||
libp2p/protobuf/minprotobuf,
|
||||
libp2p/stream/connection,
|
||||
stew/results, metrics
|
||||
stew/results, metrics,
|
||||
../waku_types
|
||||
|
||||
{.push raises: [Defect].}
|
||||
|
||||
|
|
@ -3,8 +3,9 @@ import
|
|||
json_rpc/rpcserver,
|
||||
nimcrypto/[sysrand, hmac, sha2],
|
||||
eth/[common, rlp, keys, p2p],
|
||||
../../../protocol/v2/waku_relay,
|
||||
../waku_types, ../wakunode2
|
||||
../../protocol/waku_relay,
|
||||
../../waku_types,
|
||||
../wakunode2
|
||||
|
||||
proc setupWakuRPC*(node: WakuNode, rpcsrv: RpcServer) =
|
||||
|
||||
|
|
@ -2,7 +2,7 @@ import
|
|||
std/options,
|
||||
eth/keys,
|
||||
eth/p2p/rlpx_protocols/whisper/whisper_types,
|
||||
./waku_types
|
||||
../waku_types
|
||||
|
||||
export whisper_types, waku_types, keys, options
|
||||
|
||||
|
|
@ -10,8 +10,9 @@ import
|
|||
libp2p/protocols/pubsub/pubsub,
|
||||
libp2p/peerinfo,
|
||||
libp2p/standard_setup,
|
||||
../../protocol/v2/[waku_relay, waku_store, waku_filter, waku_swap, message_notifier],
|
||||
./waku_types, ./message_store
|
||||
../protocol/[waku_relay, waku_store, waku_filter, waku_swap, message_notifier],
|
||||
../waku_types,
|
||||
./message_store
|
||||
|
||||
export waku_types
|
||||
|
||||
|
|
@ -349,7 +350,8 @@ proc connectToNodes*(n: WakuNode, nodes: seq[PeerInfo]) {.async.} =
|
|||
when isMainModule:
|
||||
import
|
||||
confutils, json_rpc/rpcserver, metrics,
|
||||
./config, ./rpc/wakurpc, ../common
|
||||
./config, ./rpc/wakurpc,
|
||||
../../common/utils/nat
|
||||
|
||||
proc startRpc(node: WakuNode, rpcIp: ValidIpAddress, rpcPort: Port) =
|
||||
let
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Waku v2 protocol
|
||||
|
||||
This folder contains implementations of [Waku v2 protocols](https://specs.vac.dev/specs/waku/v2/waku-v2.html).
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
import
|
||||
std/tables,
|
||||
chronos,
|
||||
./../../node/v2/waku_types
|
||||
../waku_types
|
||||
|
||||
# The Message Notification system is a method to notify various protocols
|
||||
# running on a node when a new message was received.
|
||||
|
|
@ -11,7 +11,7 @@ import
|
|||
libp2p/crypto/crypto,
|
||||
libp2p/switch,
|
||||
./message_notifier,
|
||||
./../../node/v2/waku_types
|
||||
../waku_types
|
||||
|
||||
# NOTE This is just a start, the design of this protocol isn't done yet. It
|
||||
# should be direct payload exchange (a la req-resp), not be coupled with the
|
||||
|
|
@ -8,7 +8,7 @@ import
|
|||
libp2p/protocols/pubsub/[pubsub, floodsub, gossipsub],
|
||||
libp2p/protocols/pubsub/rpc/messages,
|
||||
libp2p/stream/connection,
|
||||
../../node/v2/waku_types
|
||||
../waku_types
|
||||
|
||||
declarePublicGauge total_messages, "number of messages received"
|
||||
|
||||
|
|
@ -8,7 +8,8 @@ import
|
|||
libp2p/protobuf/minprotobuf,
|
||||
libp2p/stream/connection,
|
||||
./message_notifier,
|
||||
./../../node/v2/[waku_types, message_store]
|
||||
../node/message_store,
|
||||
../waku_types
|
||||
|
||||
logScope:
|
||||
topics = "wakustore"
|
||||
|
|
@ -31,7 +31,7 @@ import
|
|||
libp2p/protobuf/minprotobuf,
|
||||
libp2p/stream/connection,
|
||||
./message_notifier,
|
||||
./../../node/v2/waku_types
|
||||
../waku_types
|
||||
|
||||
export waku_types
|
||||
|
||||
Loading…
Reference in New Issue