logos-messaging-nim/examples/lightpush_mix/lightpush_publisher_mix.nim
Igor Sirotin 8125cd0262
refactor(metrics): give every metric a logos_delivery_ prefix (#4074)
* refactor(metrics): prefix node metrics with logos_delivery_

Every metric the node exports now starts with logos_delivery_. There was no
prefix mechanism before: nim-metrics derives the exported name from the Nim
identifier, no declaration passed an explicit `name = "..."`, and the waku_
convention was maintained by hand -- 69 of the 80 node metrics followed it and
11 did not (query_count, query_time_secs, event_loop_load,
event_loop_accumulated_lag_secs, postgres_payload_size_bytes, reconciliation_*,
total_* and the camelCase rendezvousPeerFoundTotal).

Identifiers are renamed rather than given a `name = "..."` argument, keeping the
invariant that the Nim identifier is the exported name and letting the compiler
check every call site.

rendezvousPeerFoundTotal becomes logos_delivery_rendezvous_peer_found: it was the
only camelCase metric, and the trailing Total was redundant since nim-metrics
already appends _total to counters at exposition time.

library/ is untouched on purpose -- `proc waku_version()` in
kernel_api/debug_node_api.nim is the exported libwaku C ABI symbol, not the gauge
of the same name in node_telemetry.nim.

BREAKING CHANGE: metric names change. Dashboards, alert rules and recording rules
that reference waku_* must be updated; see docs/operators/how-to/monitor.md.

* refactor(metrics): prefix auxiliary app metrics with logos_delivery_

Applies the same prefix to the tools shipped from this repo: liteprotocoltester
(lpt_*), networkmonitor (networkmonitor_*), chat2bridge (chat2_*) and the
lightpush_mix example (lp_mix_*).

These tools are not the delivery node and already had their own consistent
prefixes, so this commit is separable from the node rename if the intent was to
namespace only the node itself.

* chore(metrics): query old and new metric names in Grafana dashboards

212 expressions across 9 dashboards now match both the waku_* and the
logos_delivery_* spelling, so panels keep working across the upgrade and over
historical data:

  sum by (type)((increase(waku_node_errors_total{...}[$__rate_interval])
              or increase(logos_delivery_node_errors_total{...}[$__rate_interval])))

The `or` is placed around the leaf, inside every aggregation. That depth is
load-bearing: `or` keeps its right operand only for label sets absent from the
left, so `sum by (type)(old) or sum by (type)(new)` aggregates each half of the
fleet separately and then discards the right one entirely -- silently dropping
every already-upgraded node. 36 panels here collapse `instance`.

Measured against a local Prometheus scraping two targets, one exporting old names
at 10/s and one exporting new names at 20/s (truth 30/s): union outside the
aggregation gives 10, union around the leaf gives 30.

Where the leaf sits in a range vector the whole call is duplicated, since
`(a or b)[5m]` is not valid PromQL.

Once every scraped node runs a release with the new names and the old samples
have aged out of retention, the `or` half can be deleted.

* test(e2e): expect logos_delivery_-prefixed metric names

The e2e suite asserts against a live /metrics endpoint, which serves only the new
names, so these are replaced rather than unioned. libp2p_* entries are unchanged.

* docs(operators): document the logos_delivery_ metric prefix

Records that every metric the node exports is prefixed, that dependency metrics
(libp2p_*, nim_gc_*, process_*) keep their own names, and shows where the `or`
has to sit if operators maintain their own dashboards or alert rules.

* refactor(metrics): name the store fleet metrics after store, not relay

logos_delivery_relay_fleet_store_msg_size_bytes and _msg_count are declared in
waku_store/protocol_metrics.nim and recorded by the store client, but carried a
relay prefix. Renamed to logos_delivery_store_fleet_msg_size_bytes and
logos_delivery_store_fleet_msg_count.

The dashboard keeps matching the old exported name, which was
waku_relay_fleet_store_*.

Note that both metrics are wrong independently of their name, see the PR
description.
2026-07-29 18:56:50 +01:00

203 lines
7.2 KiB
Nim

import
std/[tables, times, sequtils, strutils],
stew/byteutils,
chronicles,
results,
chronos,
confutils,
libp2p/crypto/crypto,
libp2p/crypto/curve25519,
libp2p_mix,
libp2p_mix/curve25519,
libp2p/multiaddress,
eth/keys,
eth/p2p/discoveryv5/enr,
metrics,
metrics/chronos_httpserver
import
logos_delivery/waku/[
common/logging,
node/peer_manager,
waku_core,
waku_core/codecs,
waku_node,
waku_enr,
discovery/waku_discv5,
factory/builder,
waku_lightpush/client,
],
./lightpush_publisher_mix_config,
./lightpush_publisher_mix_metrics
const clusterId = 66
const shardId = @[0'u16]
const
LightpushPubsubTopic = PubsubTopic("/waku/2/rs/66/0")
LightpushContentTopic = ContentTopic("/examples/1/light-pubsub-mix-example/proto")
proc splitPeerIdAndAddr(maddr: string): (string, string) =
let parts = maddr.split("/p2p/")
if parts.len != 2:
error "Invalid multiaddress format", parts = parts
return
let
address = parts[0]
peerId = parts[1]
return (address, peerId)
proc setupAndPublish(rng: crypto.Rng, conf: LightPushMixConf) {.async.} =
# use notice to filter all waku messaging
setupLog(logging.LogLevel.DEBUG, logging.LogFormat.TEXT)
notice "starting publisher", wakuPort = conf.port
let
nodeKey = crypto.PrivateKey.random(Secp256k1, rng).get()
ip = parseIpAddress("0.0.0.0")
flags = CapabilitiesBitfield.init(relay = true)
let relayShards = RelayShards.init(clusterId, shardId).valueOr:
error "Relay shards initialization failed", error = error
quit(QuitFailure)
var enrBuilder = EnrBuilder.init(nodeKey)
enrBuilder.withWakuRelaySharding(relayShards).expect(
"Building ENR with relay sharding failed"
)
let record = enrBuilder.build().valueOr:
error "failed to create enr record", error = error
quit(QuitFailure)
setLogLevel(logging.LogLevel.TRACE)
var builder = WakuNodeBuilder.init()
builder.withNodeKey(nodeKey)
builder.withRecord(record)
builder.withNetworkConfigurationDetails(ip, Port(conf.port)).tryGet()
let node = builder.build().tryGet()
node.mountMetadata(clusterId, shardId).expect(
"failed to mount waku metadata protocol"
)
node.mountLightPushClient()
try:
await node.mountPeerExchange(Opt.some(uint16(clusterId)))
except CatchableError:
error "failed to mount waku peer-exchange protocol",
error = getCurrentExceptionMsg()
return
let (destPeerAddr, destPeerId) = splitPeerIdAndAddr(conf.destPeerAddr)
let (pxPeerAddr, pxPeerId) = splitPeerIdAndAddr(conf.pxAddr)
info "dest peer address", destPeerAddr = destPeerAddr, destPeerId = destPeerId
info "peer exchange address", pxPeerAddr = pxPeerAddr, pxPeerId = pxPeerId
let pxPeerInfo =
RemotePeerInfo.init(destPeerId, @[MultiAddress.init(destPeerAddr).get()])
node.peerManager.addServicePeer(pxPeerInfo, WakuPeerExchangeCodec)
let pxPeerInfo1 =
RemotePeerInfo.init(pxPeerId, @[MultiAddress.init(pxPeerAddr).get()])
node.peerManager.addServicePeer(pxPeerInfo1, WakuPeerExchangeCodec)
if not conf.mixDisabled:
let (mixPrivKey, mixPubKey) = generateKeyPair().valueOr:
error "failed to generate mix key pair", error = error
return
(await node.mountMix(clusterId, mixPrivKey, conf.mixnodes)).isOkOr:
error "failed to mount waku mix protocol: ", error = $error
return
let dPeerId = PeerId.init(destPeerId).valueOr:
error "Failed to initialize PeerId", error = error
return
await node.mountRendezvousClient(clusterId)
await node.start()
node.peerManager.start()
node.startPeerExchangeLoop()
try:
startMetricsHttpServer("0.0.0.0", Port(8008))
except Exception:
error "failed to start metrics server: ", error = getCurrentExceptionMsg()
(await node.fetchPeerExchangePeers()).isOkOr:
warn "Cannot fetch peers from peer exchange", cause = error
if not conf.mixDisabled:
while node.getMixNodePoolSize() < conf.minMixPoolSize:
info "waiting for mix nodes to be discovered",
currentpoolSize = node.getMixNodePoolSize()
await sleepAsync(1000)
notice "publisher service started with mix node pool size ",
currentpoolSize = node.getMixNodePoolSize()
var i = 0
while i < conf.numMsgs:
var conn: Connection
if conf.mixDisabled:
let connOpt = await node.peerManager.dialPeer(dPeerId, WakuLightPushCodec)
if connOpt.isNone():
error "failed to dial peer with WakuLightPushCodec", target_peer_id = dPeerId
return
conn = connOpt.get()
else:
conn = node.wakuMix.toConnection(
MixDestination.exitNode(dPeerId), # destination lightpush peer
WakuLightPushCodec, # protocol codec which will be used over the mix connection
MixParameters(expectReply: Opt.some(true), numSurbs: Opt.some(byte(1))),
# mix parameters indicating we expect a single reply
).valueOr:
error "failed to create mix connection", error = error
return
i = i + 1
let text =
"""Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam venenatis magna ut tortor faucibus, in vestibulum nibh commodo. Aenean eget vestibulum augue. Nullam suscipit urna non nunc efficitur, at iaculis nisl consequat. Mauris quis ultrices elit. Suspendisse lobortis odio vitae laoreet facilisis. Cras ornare sem felis, at vulputate magna aliquam ac. Duis quis est ultricies, euismod nulla ac, interdum dui. Maecenas sit amet est vitae enim commodo gravida. Proin vitae elit nulla. Donec tempor dolor lectus, in faucibus velit elementum quis. Donec non mauris eu nibh faucibus cursus ut egestas dolor. Aliquam venenatis ligula id velit pulvinar malesuada. Vestibulum scelerisque, justo non porta gravida, nulla justo tempor purus, at sollicitudin erat erat vel libero.
Fusce nec eros eu metus tristique aliquet.
This is message #""" &
$i & """ sent from a publisher using mix. End of transmission."""
let message = WakuMessage(
payload: toBytes(text), # content of the message
contentTopic: LightpushContentTopic, # content topic to publish to
ephemeral: true, # tell store nodes to not store it
timestamp: getNowInNanosecondTime(),
) # current timestamp
let res = await node.wakuLightpushClient.publish(
Opt.some(LightpushPubsubTopic), message, conn
)
let startTime = getNowInNanosecondTime()
(
await node.wakuLightpushClient.publishWithConn(
LightpushPubsubTopic, message, conn, dPeerId
)
).isOkOr:
error "failed to publish message via mix", error = error.desc
logos_delivery_lp_mix_failed.inc(labelValues = ["publish_error"])
return
let latency = float64(getNowInNanosecondTime() - startTime) / 1_000_000.0
logos_delivery_lp_mix_latency.observe(latency)
logos_delivery_lp_mix_success.inc()
notice "published message",
text = text,
timestamp = message.timestamp,
latency = latency,
psTopic = LightpushPubsubTopic,
contentTopic = LightpushContentTopic
if conf.mixDisabled:
await conn.close()
await sleepAsync(conf.msgIntervalMilliseconds)
info "Sent all messages via mix"
quit(0)
when isMainModule:
let conf = LightPushMixConf.load()
let rng = crypto.newRng()
asyncSpawn setupAndPublish(rng, conf)
runForever()