42e0aa43d1
* persistency: per-job SQLite-backed storage layer (singleton, brokered)
Adds a backend-neutral CRUD library at waku/persistency/, plus the
nim-brokers dependency swap that enables it.
Architecture (ports-and-adapters):
* Persistency: process-wide singleton, one root directory.
* Job: one tenant, one DB file, one worker thread, one BrokerContext.
* Backend: SQLite via waku/common/databases/db_sqlite. Uniform schema
kv(category BLOB, key BLOB, payload BLOB) PRIMARY KEY (category, key)
WITHOUT ROWID, WAL mode.
* Writes are fire-and-forget via EventBroker(mt) PersistEvent.
* Reads are async via five RequestBroker(mt) shapes (KvGet, KvExists,
KvScan, KvCount, KvDelete). Reads return Result[T, PersistencyError].
* One storage thread per job; tenants isolated by BrokerContext.
Public surface (waku/persistency/persistency.nim):
Persistency.instance(rootDir) / Persistency.instance() / Persistency.reset()
p.openJob(id) / p.closeJob(id) / p.dropJob(id) / p.close()
p.job(id) / p[id] / p.hasJob(id)
Writes (Job form & string-id form, fire-and-forget):
persist / persistPut / persistDelete / persistEncoded
Reads (Job form & string-id form, async Result):
get / exists / scan / scanPrefix / count / deleteAcked
Key & payload encoding (keys.nim, payload.nim):
* encodePart family + variadic key(...) / payload(...) macros +
single-value toKey / toPayload.
* Primitives: string and openArray[byte] are 2-byte BE length + bytes;
int{8..64} are sign-flipped 8-byte BE; uint{16..64} are 8-byte BE;
bool/byte/char are 1 byte; enums are int64(ord(v)).
* Generic encodePart[T: tuple | object] recurses through fields() so
any composite Nim type is encodable without ceremony.
* Stable across Nim/C compiler upgrades: no sizeof, no memcpy, no
cast on pointers, no host-endianness dependency.
* `rawKey(bytes)` + `persistPut(..., openArray[byte])` let callers
bypass the built-in encoder with their own format (CBOR, protobuf...).
Lifecycle:
* Persistency.new is private; Persistency.instance is the only public
constructor. Same rootDir is idempotent; conflicting rootDir is
peInvalidArgument. Persistency.reset for test/restart paths.
* openJob opens-or-creates the per-job SQLite file; an existing file
is reused with its data preserved.
* Teardown integration: Persistency.instance registers a Teardown
MultiRequestBroker provider that closes all jobs and clears the
singleton slot when Waku.stop() issues Teardown.request.
Internal layering:
types.nim pure value types (Key, KeyRange, KvRow, TxOp,
PersistencyError)
keys.nim encodePart primitives + key(...) macro
payload.nim toPayload + payload(...) macro
schema.nim CREATE TABLE + connection pragmas + user_version
backend_sqlite.nim KvBackend, applyOps (single source of write SQL),
getOne/existsOne/deleteOne, scanRange (asc/desc,
half-open ranges, open-ended stop), countRange
backend_comm.nim EventBroker(mt) PersistEvent + 5 RequestBroker(mt)
declarations; encodeErr/decodeErr boundary helpers
backend_thread.nim startStorageThread / stopStorageThread (shared
allocShared0 arg, cstring dbPath, atomic
ready/shutdown flags); per-thread provider
registration
persistency.nim Persistency + Job types, singleton state, public
facade
../requests/lifecycle_requests.nim
Teardown MultiRequestBroker
Tests (69 cases, all passing):
test_keys.nim sort-order invariants (length-prefix strings,
sign-flipped ints, composite tuples, prefix
range)
test_backend.nim round-trip / replace / delete-return-value /
batched atomicity / asc-desc-half-open-open-
ended scans / category isolation / batch
txDelete
test_lifecycle.nim open-or-create rootDir / non-dir collision /
reopen across sessions / idempotent openJob /
two-tenant parallel isolation / closeJob joins
worker / dropJob removes file / acked delete
test_facade.nim put-then-get / atomic batch / scanPrefix
asc/desc / deleteAcked hit-miss /
fire-and-forget delete / two-tenant facade
isolation
test_encoding.nim tuple/named-tuple/object keys, embedded Key,
enum encoding, field-major composite sort,
payload struct encoding, end-to-end struct
round-trip through SQLite
test_string_lookup.nim peJobNotFound semantics / hasJob / subscript /
persistPut+get via id / reads short-circuit /
writes drop+warn / persistEncoded via id /
scan parity Job-ref vs id
test_singleton.nim idempotent same-rootDir / different-rootDir
rejection / no-arg instance lifecycle / reset
retargets / reset idempotence / Teardown.request
end-to-end
Prerequisite delivered in the same series: replace the in-tree broker
implementation with the external nim-brokers package; update all
broker call-sites (waku_filter_v2, waku_relay, waku_rln_relay,
delivery_service, peer_manager, requests/*, factory/*, api tests, etc.)
to the new package API; chat2 made to compile again.
Note: SDS adapter (Phase 5 of the design) is deferred -- nim-sds is
still developed side-by-side and the persistency layer is intentionally
SDS-agnostic.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* persistency: pin nim-brokers by URL+commit (workaround for stale registry)
The bare `brokers >= 2.0.1` form cannot resolve on machines where the
local nimble SAT solver enumerates only the registry-recorded 0.1.0 for
brokers. The nim-lang/packages entry for `brokers` carries no per-tag
metadata (only the URL), so until that registry entry is refreshed the
SAT solver clamps the available-versions list to 0.1.0 and rejects the
>= 2.0.1 constraint -- even though pkgs2 and pkgcache both have v2.0.1
cloned locally.
Pinning by URL+commit bypasses the registry path entirely. Inline
comment in waku.nimble documents the situation and the path back to
the bare form once nim-lang/packages is updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* persistency: nph format pass
Run `nph` on all 57 Nim files touched by this PR. Pure formatting:
17 files re-styled, no semantic change. Suite still 69/69.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Fix build, add local-storage-path config, lazy init of Persistency from Waku start
* fix: fix nix deps
* fixes for nix build, regenerate deps
* reverting accidental dependency changes
* Fixing deps
* Apply suggestions from code review
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* persistency tests: migrate to suite / asyncTest / await
Match the in-tree test convention (procSuite -> suite, sync test +
waitFor -> asyncTest + await):
- procSuite "X": -> suite "X":
- For tests doing async work: test -> asyncTest, waitFor -> await.
- Poll helpers (proc waitFor(t: Job, ...) in test_lifecycle.nim,
proc waitUntilExists(...) in test_facade.nim and
test_string_lookup.nim) -> Future[bool] {.async.}, internal
`waitFor X` -> `await X`, internal `sleep(N)` ->
`await sleepAsync(chronos.milliseconds(N))`.
- Renamed test_lifecycle.nim's helper proc from `waitFor(t: Job, ...)`
-> `pollExists(t: Job, ...)`; the previous name shadowed
chronos.waitFor in the chronos macro expansion.
- `chronos.milliseconds(N)` explicitly qualified because `std/times`
also exports `milliseconds` (returning TimeInterval, not Duration).
- `check await x` -> `let okN = await x; check okN` to dodge chronos's
"yield in expr not lowered" with await-as-macro-argument.
- `(await x).foo()` -> `let awN = await x; ... awN.foo() ...` for the
same reason.
waku/persistency/persistency.nim: nph also pulled the proc signatures
across multiple lines; restored explicit `Future[void] {.async.}`
return types after the colon (an intermediate nph pass had elided them).
Suite: 71 / 71 OK against the new async write surface.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* use idiomatic valueOr instead of ifs
* Reworked persistency shutdown, remove not necessary teardown mechanism
* Use const for DefaultStoragePath
* format to follow coding guidelines - no use of result and explicit returns - no functional change
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Waku
This folder contains code related to Waku, both as a node and as a protocol.
Introduction
This is an implementation in Nim of the Waku suite of protocols.
See specifications.
How to Build & Run
Prerequisites
- GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer.
Wakunode binary
# The first `make` invocation will update all Git submodules.
# You'll run `make update` after each `git pull`, in the future, to keep those submodules up to date.
make wakunode2
# See available command line options
./build/wakunode2 --help
# Connect the client directly with the Status test fleet
# TODO NYI
#./build/wakunode2 --log-level:debug --discovery:off --fleet:test --log-metrics
Note: building wakunode2 requires 2GB of RAM. The build will fail on systems not fulfilling this requirement.
Setting up a wakunode2 on the smallest digital ocean droplet, you can either
- compile on a stronger droplet featuring the same CPU architecture and downgrade after compiling, or
- activate swap on the smallest droplet, or
- use Docker.
Waku Protocol Test Suite
# Run all the Waku tests
make test
To run a specific test.
# Get a shell with the right environment variables set
./env.sh bash
# Run a specific test
nim c -r ./tests/test_waku_filter_legacy.nim
You can also alter compile options. For example, if you want a less verbose output you can do the following. For more, refer to the compiler flags and chronicles documentation.
nim c -r -d:chronicles_log_level=WARN --verbosity=0 --hints=off ./tests/waku_filter_v2/test_waku_filter.nim
You may also want to change the outdir to a folder ignored by git.
nim c -r -d:chronicles_log_level=WARN --verbosity=0 --hints=off --outdir=build ./tests/waku_filter_v2/test_waku_filter.nim
Waku Protocol Example
There are basic examples of both publishing and subscribing,
more limited in features and configuration than the wakunode2 binary,
located in examples/.
There is also a more full featured example in apps/chat2/.
Using Metrics
Metrics are available for Waku nodes.
make wakunode2
./build/wakunode2 --metrics-server
Ensure your Prometheus config prometheus.yml contains the targets you care about, e.g.:
scrape_configs:
- job_name: "waku"
static_configs:
- targets: ['localhost:8008', 'localhost:8009', 'localhost:8010']
For visualisation, similar steps can be used as is written down for Nimbus here.
There is a similar example dashboard that includes visualisation of the
envelopes available at metrics/waku-grafana-dashboard.json.
Spec support
All Waku RFCs reside at rfc.vac.dev.
Note that Waku specs are titled WAKU2-XXX
to differentiate them from a previous legacy version of Waku with RFC titles in the format WAKU-XXX.
The legacy Waku protocols are stable, but not under active development.
Generating and configuring a private key
By default a node will generate a new, random key pair each time it boots,
resulting in a different public libp2p multiaddrs after each restart.
To maintain consistent addressing across restarts,
it is possible to configure the node with a previously generated private key using the --nodekey option.
wakunode2 --nodekey=<64_char_hex>
This option takes a Secp256k1 private key in 64 char hexstring format.
To generate such a key on Linux systems,
use the openssl rand command to generate a pseudo-random 32 byte hexstring.
openssl rand -hex 32
Example output:
$ openssl rand -hex 32
6a29e767c96a2a380bb66b9a6ffcd6eb54049e14d796a1d866307b8beb7aee58
where the key 6a29e767c96a2a380bb66b9a6ffcd6eb54049e14d796a1d866307b8beb7aee58 can be used as nodekey.
To create a reusable keyfile on Linux using openssl,
use the ecparam command coupled with some standard utilities
whenever you want to extract the 32 byte private key in hex format.
# Generate keyfile
openssl ecparam -genkey -name secp256k1 -out my_private_key.pem
# Extract 32 byte private key
openssl ec -in my_private_key.pem -outform DER | tail -c +8 | head -c 32| xxd -p -c 32
Example output:
read EC key
writing EC key
0c687bb8a7984c770b566eae08520c67f53d302f24b8d4e5e47cc479a1e1ce23
where the key 0c687bb8a7984c770b566eae08520c67f53d302f24b8d4e5e47cc479a1e1ce23 can be used as nodekey.
wakunode2 --nodekey=0c687bb8a7984c770b566eae08520c67f53d302f24b8d4e5e47cc479a1e1ce23
Configuring a domain name
It is possible to configure an IPv4 DNS domain name that resolves to the node's public IPv4 address.
wakunode2 --dns4-domain-name=mynode.example.com
This allows for the node's publicly announced multiaddrs to use the /dns4 scheme.
In addition, nodes with domain name and secure websocket configured,
will generate a discoverable ENR containing the /wss multiaddr with /dns4 domain name.
This is necessary to verify domain certificates when connecting to this node over secure websocket.
Using DNS discovery to connect to existing nodes
A node can discover other nodes to connect to using DNS-based discovery. The following command line options are available:
--dns-discovery Enable DNS Discovery
--dns-discovery-url URL for DNS node list in format 'enrtree://<key>@<fqdn>'
--dns-addrs-name-server DNS name server IPs to query. Argument may be repeated.
--dns-discoveryis used to enable DNS discovery on the node. Waku DNS discovery is disabled by default.--dns-discovery-urlis mandatory if DNS discovery is enabled. It contains the URL for the node list. The URL must be in the formatenrtree://<key>@<fqdn>where<fqdn>is the fully qualified domain name and<key>is the base32 encoding of the compressed 32-byte public key that signed the list at that location.
A node will attempt connection to all discovered nodes.
This can be used, for example, to connect to one of the existing fleets. Current URLs for the published fleet lists:
- production fleet:
enrtree://AIRVQ5DDA4FFWLRBCHJWUWOO6X6S4ZTZ5B667LQ6AJU6PEYDLRD5O@sandbox.waku.nodes.status.im - test fleet:
enrtree://AOGYWMBYOUIMOENHXCHILPKY3ZRFEULMFI4DOM442QSZ73TT2A7VI@test.waku.nodes.status.im
See the separate tutorial for a complete guide to DNS discovery.
Enabling Websocket
Websocket is currently the only Waku transport supported by browser nodes that uses js-waku. Setting up websocket enables your node to directly serve browser peers.
A valid certificate is necessary to serve browser nodes,
you can use letsencrypt:
sudo letsencrypt -d <your.domain.name>
You will need the privkey.pem and fullchain.pem files.
To enable secure websocket, pass the generated files to wakunode2:
Note, the default port for websocket is 8000.
wakunode2 --websocket-secure-support=true --websocket-secure-key-path="<letsencrypt cert dir>/privkey.pem" --websocket-secure-cert-path="<letsencrypt cert dir>/fullchain.pem"
Self-signed certificates
Self-signed certificates are not recommended for production setups because:
- Browsers do not accept self-signed certificates
- Browsers do not display an error when rejecting a certificate for websocket.
However, they can be used for local testing purposes:
mkdir -p ./ssl_dir/
openssl req -x509 -newkey rsa:4096 -keyout ./ssl_dir/key.pem -out ./ssl_dir/cert.pem -sha256 -nodes
wakunode2 --websocket-secure-support=true --websocket-secure-key-path="./ssl_dir/key.pem" --websocket-secure-cert-path="./ssl_dir/cert.pem"