8125cd0262
* refactor(metrics): prefix node metrics with logos_delivery_
Every metric the node exports now starts with logos_delivery_. There was no
prefix mechanism before: nim-metrics derives the exported name from the Nim
identifier, no declaration passed an explicit `name = "..."`, and the waku_
convention was maintained by hand -- 69 of the 80 node metrics followed it and
11 did not (query_count, query_time_secs, event_loop_load,
event_loop_accumulated_lag_secs, postgres_payload_size_bytes, reconciliation_*,
total_* and the camelCase rendezvousPeerFoundTotal).
Identifiers are renamed rather than given a `name = "..."` argument, keeping the
invariant that the Nim identifier is the exported name and letting the compiler
check every call site.
rendezvousPeerFoundTotal becomes logos_delivery_rendezvous_peer_found: it was the
only camelCase metric, and the trailing Total was redundant since nim-metrics
already appends _total to counters at exposition time.
library/ is untouched on purpose -- `proc waku_version()` in
kernel_api/debug_node_api.nim is the exported libwaku C ABI symbol, not the gauge
of the same name in node_telemetry.nim.
BREAKING CHANGE: metric names change. Dashboards, alert rules and recording rules
that reference waku_* must be updated; see docs/operators/how-to/monitor.md.
* refactor(metrics): prefix auxiliary app metrics with logos_delivery_
Applies the same prefix to the tools shipped from this repo: liteprotocoltester
(lpt_*), networkmonitor (networkmonitor_*), chat2bridge (chat2_*) and the
lightpush_mix example (lp_mix_*).
These tools are not the delivery node and already had their own consistent
prefixes, so this commit is separable from the node rename if the intent was to
namespace only the node itself.
* chore(metrics): query old and new metric names in Grafana dashboards
212 expressions across 9 dashboards now match both the waku_* and the
logos_delivery_* spelling, so panels keep working across the upgrade and over
historical data:
sum by (type)((increase(waku_node_errors_total{...}[$__rate_interval])
or increase(logos_delivery_node_errors_total{...}[$__rate_interval])))
The `or` is placed around the leaf, inside every aggregation. That depth is
load-bearing: `or` keeps its right operand only for label sets absent from the
left, so `sum by (type)(old) or sum by (type)(new)` aggregates each half of the
fleet separately and then discards the right one entirely -- silently dropping
every already-upgraded node. 36 panels here collapse `instance`.
Measured against a local Prometheus scraping two targets, one exporting old names
at 10/s and one exporting new names at 20/s (truth 30/s): union outside the
aggregation gives 10, union around the leaf gives 30.
Where the leaf sits in a range vector the whole call is duplicated, since
`(a or b)[5m]` is not valid PromQL.
Once every scraped node runs a release with the new names and the old samples
have aged out of retention, the `or` half can be deleted.
* test(e2e): expect logos_delivery_-prefixed metric names
The e2e suite asserts against a live /metrics endpoint, which serves only the new
names, so these are replaced rather than unioned. libp2p_* entries are unchanged.
* docs(operators): document the logos_delivery_ metric prefix
Records that every metric the node exports is prefixed, that dependency metrics
(libp2p_*, nim_gc_*, process_*) keep their own names, and shows where the `or`
has to sit if operators maintain their own dashboards or alert rules.
* refactor(metrics): name the store fleet metrics after store, not relay
logos_delivery_relay_fleet_store_msg_size_bytes and _msg_count are declared in
waku_store/protocol_metrics.nim and recorded by the store client, but carried a
relay prefix. Renamed to logos_delivery_store_fleet_msg_size_bytes and
logos_delivery_store_fleet_msg_count.
The dashboard keeps matching the old exported name, which was
waku_relay_fleet_store_*.
Note that both metrics are wrong independently of their name, see the PR
description.
Examples
Compile
Make all examples.
make example2
Waku API
Uses the simplified Waku API to create and start a node, you need an RPC endpoint for Linea Sepolia for RLN:
./build/waku_api --ethRpcEndpoint=https://linea-sepolia.infura.io/v3/<your key>
If you can't be bothered but still want to see some action, just run the binary and it will use a non-RLN network:
./build/waku_api
## publisher/subscriber
Within examples/ you can find a publisher and a subscriber. The first one publishes messages to the default pubsub topic on a given content topic, and the second one runs forever listening to that pubsub topic and printing the content it receives.
Some notes:
- These examples are meant to work even if you are behind a firewall and you can't be discovered by discv5.
- You only need to provide a reachable bootstrap peer (see our fleets)
- The examples are meant to work out of the box.
- Note that both services wait for some time until a given minimum amount of connections are reached. This is to ensure messages are gossiped.
Run:
Wait until the subscriber is ready.
./build/subscriber
And run a publisher
./build/publisher
See how the subscriber received the messages published by the publisher. Feel free to experiment from different machines in different locations.
resource-restricted publisher/subscriber (lightpush/filter)
To illustrate publishing and receiving messages on a resource-restricted client,
examples/v2 also provides a lightpush_publisher and a filter_subscriber.
The lightpush_publisher continually publishes messages via a lightpush service node
to the default pubsub topic on a given content topic.
The filter_subscriber subscribes via a filter service node
to the same pubsub and content topic.
It runs forever, maintaining this subscription
and printing the content it receives.
Run Start the filter subscriber.
./build/filter_subscriber
And run a lightpush publisher
./build/lightpush_publisher
See how the filter subscriber receives messages published by the lightpush publisher.
Neither the publisher nor the subscriber participates in relay,
but instead make use of service nodes to save resources.
Feel free to experiment from different machines in different locations.