nwaku/waku/v2
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
..
node Refactoring timestamps (#842) 2022-02-17 16:00:15 +01:00
protocol Refactoring timestamps (#842) 2022-02-17 16:00:15 +01:00
utils Refactoring timestamps (#842) 2022-02-17 16:00:15 +01:00
README.md Update websocket documentation to use domain certificates (#848) 2022-02-15 17:07:55 +11:00

README.md

Waku v2

This folder contains code related to Waku v1, both as a node and as a protocol.

Introduction

This is an implementation in Nim of Waku v2, which is currently in draft/beta stage.

See spec.

How to Build & Run

Prerequisites

  • GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.

Wakunode

# The first `make` invocation will update all Git submodules.
# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
make wakunode2

# See available command line options
./build/wakunode2 --help

# Connect the client directly with the Status test fleet
# TODO NYI
#./build/wakunode2 --log-level:debug --discovery:off --fleet:test --log-metrics

Waku v2 Protocol Test Suite

# Run all the Waku v2 tests
make test2

You can also run a specific test (and alter compile options as you want):

# Get a shell with the right environment variables set
./env.sh bash
# Run a specific test
nim c -r ./tests/v2/test_waku_filter.nim

Waku v2 Protocol Example

There is a more basic example, more limited in features and configuration than the wakunode1, located in examples/v2/basic2.nim.

There is also a more full featured example in examples/v2/chat2.nim.

Waku Quick Simulation

NOTE: This section might be slightly out of date as it was written for Waku v1.

One can set up several nodes, get them connected and then instruct them via the JSON-RPC interface. This can be done via e.g. web3.js, nim-web3 (needs to be updated) or simply curl your way out.

The JSON-RPC interface is currently the same as the one of Whisper. The only difference is the addition of broadcasting the topics interest when a filter with a certain set of topics is subcribed.

The quick simulation uses this approach, start_network launches a set of wakunodes, and quicksim instructs the nodes through RPC calls.

Example of how to build and run:

# Build wakunode + quicksim with metrics enabled
make NIMFLAGS="-d:insecure" wakusim2

# Start the simulation nodes, this currently requires multitail to be installed
# TODO Partial support for Waku v2
./build/start_network2 --topology:FullMesh --amount:6 --test-node-peers:2
# In another shell run
./build/quicksim2

The start_network2 tool will also provide a prometheus.yml with targets set to all simulation nodes that are started. This way you can easily start prometheus with this config, e.g.:

cd ./metrics/prometheus
prometheus --config.file=prometheus.yml

A Grafana dashboard containing the example dashboard for each simulation node is also generated and can be imported in case you have Grafana running. This dashboard can be found at ./metrics/waku-sim-all-nodes-grafana-dashboard.json

To read more details about metrics, see next section.

Using Metrics

Metrics are available for valid envelopes and dropped envelopes.

To compile in an HTTP endpoint for accessing the metrics we need to provide the insecure flag:

make NIMFLAGS="-d:insecure" wakunode2
./build/wakunode2 --metrics-server

Ensure your Prometheus config prometheus.yml contains the targets you care about, e.g.:

scrape_configs:
  - job_name: "waku"
    static_configs:
      - targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']

For visualisation, similar steps can be used as is written down for Nimbus here.

There is a similar example dashboard that includes visualisation of the envelopes available at metrics/waku-grafana-dashboard.json.

Spec support

This section last updated November 16, 2020

All Waku v2 specs, except for bridge, are currently in draft.

Docker Image

By default, the target will be a docker image with wakunode, which is the Waku v1 node. You can change this to wakunode2, the Waku v2 node like this:

make docker-image MAKE_TARGET=wakunode2
docker run --rm -it statusteam/nim-waku:latest --help

Enabling Websocket

Websocket is currently the only Waku transport supported by browser nodes that uses js-waku. Setting up websocket enables your node to directly serve browser peers.

A valid certificate is necessary to serve browser nodes, you can use letsencrypt:

sudo letsencrypt -d <your.domain.name>

You will need the privkey.pem and fullchain.pem files.

To enable secure websocket, pass the generated files to wakunode2: Note, the default port for websocket is 8000.

wakunode2 --websocket-secure-support=true --websocket-secure-key-path="<letsencrypt cert dir>/privkey.pem" --websocket-secure-cert-path="<letsencrypt cert dir>/fullchain.pem"

Self-signed certificates

Self-signed certificates are not recommended for production setups because:

  • Browsers do not accept self-signed certificates
  • Browsers do not display an error when rejecting a certificate for websocket.

However, they can be used for local testing purposes:

mkdir -p ./ssl_dir/
openssl req -x509 -newkey rsa:4096 -keyout ./ssl_dir/key.pem -out ./ssl_dir/cert.pem -sha256 -nodes
wakunode2 --websocket-secure-support=true --websocket-secure-key-path="./ssl_dir/key.pem" --websocket-secure-cert-path="./ssl_dir/cert.pem"